Skip to content

2026.11.21.1: the test and CI design (mtest 0.11), the home's unit cache, offline builds and the newest defaults, package features, build programs' observed inputs (mcpp issues) - #984

Merged
speak-agent merged 77 commits into
mainfrom
fix/rc-include-dir-globs
Nov 21, 2026

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Release 2026.11.21.1: the test and CI architecture of design 2026-11-20 with its three sub-designs of 2026-11-21 (build programs' observed inputs, offline builds with host inputs and the toolchain defaults, package features), in one pull request. The CHANGELOG section [2026.11.21.1] is the release's notes. Refs #986, #976, #927, #802, #293, #988, #989, #990, #991, #852, #992.

Engine

Tests

mtest 0.11.2. The kit records each command's steps and the surface it uses, shares one home as a run fixture, gives a case that changes a home its own, declares diagnostic lines and compares contractual outputs with goldens. The surface ledger covers every item of mcpp emit surface by a passing case or a waiver with its reason; the CPU budget is a gate.

CI

Tiers reach the jobs; a full run that passes leaves verified-<tree>; a push to main of a verified tree registers that run instead of testing again; release-build.yml holds the four release builds; release.yml promotes a verified run's archives. docs/92 and the release skill state the procedure.

Specs, usage chapters (en and zh), design records and the CHANGELOG are updated with the change.

…ded, not as globs

A relative include_dirs entry is a glob; the compile steps expand it (package_root.cpp), but the resource compiler received it as written. On Windows LLVM's tools expand wildcards in their own command line against files, so llvm-windres's preprocessor saw several inputs and refused with "cannot specify -o when generating multiple output files". compat.protobuf's protoc host tool failed so on windows-2025, which blocked gRPC's code generation there. The resources e2e case 837 gains R4: a member's `*/inc` reaches the resource compiler expanded.
…ws_resources stays under the 400-line limit)
…st's are, keeping a literal directory a build step creates later

resource_include_dirs called expand_dir_glob, which drops a literal entry
that does not exist when the build is planned; before mcpp#984 such an entry
was joined as written. It now calls expand_manifest_include_entry, the
expansion the root manifest's compile steps use, which keeps it (#390).
e2e 837 states R5: gui's generated/inc reaches the rc_object edge.
…its units (#986); each cache refusal names its reason

The instance digest hashed each unit's object path, so mcpp test, which
keeps a unit an inactive feature gates, gave the package a second instance
and compiled every unit again. It now hashes the package's distinct unit
configurations and its imports' digests. A package's include directories
have one expansion, expand_manifest_include_entry, for its compile steps,
its private part and its resource script.

serve_or_fill names why a package is not served (no-identity,
unnamed-sources, unnamed-upstream, outside-store, unaddressable-unit, miss)
under --verbose and in the build context. The token found why
mcpplibs.mtest was never cached: its synthesized entry main had no cache
address, which took the whole package out. An entry unit now compiles with
the image that links it, and the package's other units are cached; a
second project's test shows mcpplibs.mtest as Cached.

e2e: 986 (the issue's reproduction), 1037 B (mtest cached in the second
project).
…mory threshold; an archive run selects case parts; SPEC-017 2.5 (#976, #927)
…tems used and limit; estimates read the runs of the machine's class (SPEC-017 P5.6, P12.4)
…rgest resident set; a test run keeps its builds; a step that runs a minute is named (#927)
…eleton's mtest and the launchers' os with them
…ables under dev ([profile.<name>] dependency_debug; #927)
… home, keyed by its contract; key paths are named by role (#986, #927)
…s contents is placed from the home's unit cache, and no compiler runs (#927)
…a BMI's producer key is stated before it is published, and a relocated attempt's BMI is not moved twice
…26-11-20 §5.16)

The surface is each command and flag of the command line's definition
(mcpplibs.cmdline 0.0.5 exports the tree), each key of the manifest's registry
and each reason token, one '<kind> <name>' per line. With MCPP_SURFACE_RECORD,
an invocation appends the command and flags it was given, the registry keys
its manifests state and the reasons it records (mcpp.surface).
A program that declares a fixture (a component event with fixture) serves it
under MTEST_SERVE before the first program whose cases use it; each user gets
its endpoint as MTEST_FIXTURE_<NAME>; the last user's end closes its input.
fixtures.ndjson states each fixture's preparation; programs.ndjson names the
fixtures a program used; a fixture that is not ready leaves its users not run,
'infra' in the store (P11.4).
… (design 2026-11-20 §5.11)

The build directory's records name the files each unit read: ninja's deps log
(format 4, read directly) and the scans' depfiles. When every unit of the
project has a record, a changed header selects the tests built from a unit
that read it, and the tests that run a program of the package when such a
unit is not a test's own; otherwise every test is affected, as before.
… the runner's new clauses

Item 17 states the instance digest as a function of the package's
configuration; 18 the roles of paths in keys and the home OUT_DIR of a
named-source package's build program; 19 [build] memory_threshold; 20
[profile.<name>] dependency_debug. SPEC-017 states mcpp's side of O6 and the
surface mcpp emits and records. self env announces test-steps, test-budget,
test-jitter and test-fixtures, which SPEC-017 names. An invalid memory
threshold names the source that holds instead.
…that read the user's as upstreams, invariants, waits, diagnostic lines, golden outputs (design 2026-11-20 §5.10, §5.16, §5.18, §5.20, §5.21, §6)

- Each mcpp command a case runs records the surface items it used
  (MCPP_SURFACE_RECORD) as the case's exercise event.
- The shared home is the run fixture 'home', filled once per run; every group
  uses it. A case whose assertion holds only while nothing else writes its home
  runs mcpp in a home of its own (own_home) instead of holding the machine.
- Inherited homes, in the kit and in _inherit_toolchain.sh, name the user's
  homes as upstreams instead of linking each payload, and copy no subos
  (mcpp#293); 914 checks that no file the upstream's entries reach is written.
- Invariant cases with bounds: an unchanged rebuild compiles nothing, and a
  dev-dependency is served from the cache to a second project.
- Waits for a time only fall (check_waits.sh, waits.baseline); five that only
  moved a modification time touch the file instead.
- The kit declares mcpp's warning and error lines diagnostic, under report.
- The machine protocol and the surface are compared whole with golden files.
…et, run fixtures, header selection, the home's caches, the build record, emit surface and the records an invocation appends; the e2e rules of the kit; the spread of functions measured in e2e-coverage
…n tested (design 2026-11-20 §5.22, §7.2)

- ci.yml: the run's tree; the tiers reach the jobs (bootstrap, cross, the target matrix, the
  upgrade, openkal, the iOS and MSVC-xlings jobs, the second macOS image and Linux's other
  toolchains run in `full`); a push to main whose tree a run verified registers that run and tests
  nothing; `verdict` judges every job and leaves `verified-<tree>` after a `full` pass;
  `merge_group` is accepted.
- release-build.yml: the four release builds, moved from release.yml. ci.yml's `full` tier runs
  them, and its Linux shards test the program of the Linux archive (use-built-mcpp `form`).
- release.yml: `find` reads the tag's tree; `promote` takes the verifying run's archives, checked
  against their checksums and version; without such a run the four builds run, and the summary
  says which.
- The unit shards come from the measured work of main's last run; build.yml restores main's unit
  cache and, on main, keeps the entries its build used.
- The report's sections and `mtest gate`; jitter.yml (weekly, by hand, and on a flip in main's
  run); the downstream action project-test; main_run.sh resolves main's registered run.
- docs/92 and the mcpp-release skill: the release pull request passes `ci:full` before it is
  merged, main's run is not cancelled, and the release promotes.
…es, measurements, and where it differs from the design
@Sunrisepeak Sunrisepeak changed the title fix: a resource script compiles with its package's include_dirs expanded, not as globs The test and CI design of 2026-11-20: mtest 0.11 in mcpp, the build's caches and records, the test kit, and CI that verifies a tree once (#986, #976, #927, #802, #293) Nov 20, 2026
…cs and the surface

Cases 1128 to 1131 state the design's T1 to T6, T9 and T10: a header edit
compiles and runs the program again through the fast path and after
target/.build-mcpp is deleted, an unrelated edit does not, a record of an
earlier engine compiles once; the header note is said once per project and
again after mcpp clean, one note for a workspace's members; a file read
outside the package is warned once, kept in the envelope with shown false,
and refused by mcpp publish, as an untracked one is; a variable two
dependencies answer is no spelling's, answered by the first under the
deprecated reading and named by the spellings that work. The unit test of
publish_by_variable states each channel's variables once.

The notices of a program are judged on every plan, so --verbose shows a
suppressed one; a row without reads makes the fast path plan. Registered
notices are items of mcpp emit surface (`notice <domain>`); the kinds of
work not yet implemented are not registered.

docs/30 states the observed and the declared inputs, "Headers or modules" and
the variable rule; docs/50 the codes, `shown` and the reason token; SPEC-007
0.16 the clauses and protocol 21; the contributing skill the sink and the
registration of a notice's policy.
…026-11-21 offline and defaults §3.6, §3.9)

A compiler's version is its driver's -dumpfullversion (GCC; else
-dumpversion) or -dumpversion (clang); the banner names the family. A
triple's vendor field is any word, as config.sub reads it, and a toolchain
whose target mcpp cannot read is a degradation (toolchain/triple).

The command line's `<package>@<version>` is read by resolve_cli_selection:
the written prefix selects the highest key it starts (mcpp.xpkg_version's
reading), `@=` one key, an operator the requirement grammar, and an exact
miss names the finer keys. `mcpp new --template` and `--list-templates` use
it. SPEC-001 1.12 §4.5 states it; docs/01 and docs/21 describe both.
…version, and companions by declaration

A partial version is the newest release of the local index in `toolchain install`,
`toolchain default` and a build alike; a partial version mcpp.toml states is recorded in
mcpp.lock (`[toolchain."xim:gcc@16"]`) and read back until `mcpp update`, and a moved default
the project follows is said once (mcpp#988, SPEC-006 §2.6 to §2.8). A toolchain's binutils,
kernel headers and runtime are the versions its payload declares, not the first directory a
walk of the store returns; `companions = { binutils = ... }` states another, and an older one
than the declared is refused (`companion-too-old`, mcpp#990, SPEC-006 §2.9).
…ds it

The post-link validation and `mcpp self doctor` looked for a required soname on the union of
the plan's search paths, the executable's DT_RPATH. An object with DT_RUNPATH, as every payload
xlings installs carries, searches no DT_RPATH, so xim:slint's libslint_cpp.so could not load
X11, Wayland or GL in a program mcpp built while both checks passed. Each required soname is
looked up from the requiring package's own shared objects by glibc's rule, through one function
both checks call (`runtime/dlopen-unreachable`); the DT_NEEDED walk applies the same rule to the
farm. The validation record moves to schema 2, so earlier verdicts are judged again.
… host input is said once, and a build does not require patchelf (mcpp#990; design 2026-11-21 offline and defaults §3.3)
…kes one into a home, [build] offline and [xlings] archives by tier, and an offline run installs from the archive directories (mcpp#990; design 2026-11-21 offline and defaults §3.2)
…ault]] data of the built-in table, and a release lists each against the index (mcpp#988; design 2026-11-21 offline and defaults §3.7)
…ry is served only against the interfaces it was compiled with; the run-wide keys are stated per plan

- Package features (design 2026-11-21, package features; SPEC-004 §16): one catalogue
  (`feature_catalog`); the manifest of an installed version read once for the build's graph and
  the queries (`mcpp.pm.package_manifest`, SPEC-001 §3.5); `mcpp search --format json` states
  each mcpp package's features for its newest version as `declared`, `not-installed` or
  `unreadable`, installing nothing; `mcpp why features [--format json]` (`mcpp.why.features`)
  from resolution.json's `features` and `feature_list`; `--protocol-version` lists
  `package-features`; a table-form feature and a descriptor's `features.<f>` accept
  `description`.
- The global cache records the module interfaces an entry was compiled against and serves it
  only against them (SPEC-001 §10.4 item 7): after `mcpp cache gc` removed an upstream's entry, a
  dependent was served against another build of that interface and failed with
  `module '<name>' CRC mismatch`.
- `[build] offline` and `allow_host_tools` enter the members' grouping key, and each plan states
  or withdraws the project's tier (SPEC-004 §9 items 10, 22).
- `mcpp test --failed` reads the newest of two runs started in the same second; the store's
  nasm linked from an upstream home is the store's, judged by path.
- toolchain.cpp is cut at the phase boundary (toolchain_resolver.cpp, P2); the first-run
  defaults are named by the host's tokens; the build-program input cases touch an edit instead
  of sleeping; the bundle asks the provider for its configuration file.
- Cases that asserted a cold cache use a home of their own or assert the package in the build;
  mcpplibs.mtest 0.11.2 throughout, the CI engine included.
@Sunrisepeak Sunrisepeak added the ci:full Run the full e2e set on this pull request (read when a push starts the run) label Nov 21, 2026
…ject's record and does not write it (the build database leaves the project's tree unchanged)
…t back (SPEC-017 O11 item 5); the offline refusal names the host's first-run toolchain from its one answer
… case states (1127); the bundle case runs where no network namespace is given; the Linux shards install the toolchains their cases probe
…ow), musl gcc 16.2.0 on other Linux architectures, ninja 1.13.2-mcpp.1 and patchelf 0.19.3; a default is the newest release every platform it serves publishes, so llvm and the musl rows stay (mcpp#988; design 2026-11-21 §3.7)
…re spelled as UTF-8; the test design records the second checkpoint's findings
…this commit pins (ninja and patchelf), read from the built-in table's defaults
…he first run's toolchain is the one the binary answers (29)
…irectory (<ns>.<name>/ before 2026.11.21.1, <ns>_<name>/<instance>/ from it, mcpp#986)
… home's environment too (gcc 16.2.0 revision 1), so an undescribed environment's link finds the home's C library
…s keeps the home's default toolchain (60); 860 measures the programs' compiles once the engine module is cached and reports the graph's steps; 955 and 182 state the time a cold cache takes
…ake_refused); messages that name a toolchain to write or install name this host's first run from the built-in table, and doctor's cross hints name toolchain install --target
…rest at the case's end; cases 1134 to 1140 and steps of 167, 234, 325, 990, 1127 exercise the surface no case used; seven waivers state why; the design records the third checkpoint
… is its configuration's, mcpp#986), so the previous release's dependency is compiled once and then served; the CHANGELOG states the one rebuild
…nuation lines (the index, what was done, the hint)
…states: the compile and link lines take -B from gcc::binutils_prefix_dir, the std module's one answer, not from the archive tool's directory (bases 925)
…one at -j 1 (each compile waits for its own scan); the mingw suite states 182's limit, which the runner reads for a selected case; the design records the scans, the limit and the -B
@speak-agent speak-agent changed the title The test and CI design of 2026-11-20: mtest 0.11 in mcpp, the build's caches and records, the test kit, and CI that verifies a tree once (#986, #976, #927, #802, #293) 2026.11.21.1: the test and CI design (mtest 0.11), the home's unit cache, offline builds and the newest defaults, package features, build programs' observed inputs (mcpp#986, #976, #927, #802, #293, #988, #989, #990, #991, #852, #992) Nov 21, 2026
@speak-agent
speak-agent merged commit 6b90f4d into main Nov 21, 2026
102 checks passed
@Sunrisepeak Sunrisepeak changed the title 2026.11.21.1: the test and CI design (mtest 0.11), the home's unit cache, offline builds and the newest defaults, package features, build programs' observed inputs (mcpp#986, #976, #927, #802, #293, #988, #989, #990, #991, #852, #992) 2026.11.21.1: the test and CI design (mtest 0.11), the home's unit cache, offline builds and the newest defaults, package features, build programs' observed inputs (mcpp issues) Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:full Run the full e2e set on this pull request (read when a push starts the run)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants