Repository navigation
feat!: v0.5.0 Wave 2 — mds-cli output surface + watch (pipe/exit contract, panic hook, typed paths, anchored writes, lint result sink, watch robustness) - #438
Draft
dean0x wants to merge 190 commits into
Conversation
Characterization goldens for `mds lint`, added before any lint or stream change so every later commit that alters lint output has to regenerate them and name the cells it changed. They deliberately pin TODAY's output, including output that is known to be wrong (a clean JSON envelope printed before a failed write, residual diagnostics rendered against the pre-fix source, a partial module's safe fix rejected, no cap notice in report-only mode, the stdin `--fix --format json` refusal as plain text). What is pinned, per cell: exit code, exact stdout, exact stderr and the source file's bytes after the run (unchanged / rewritten bytes / missing). Matrix: input (stdin with cwd = the fixture dir so mds.json applies, or a relative `x.mds`) x format (human, json) x --quiet (off, on) x fix mode (none, --fix, --fix --check, --fix --diff, --fix --check --diff) x 13 fixtures: clean, warn, error (mds.json raises unused-variable), fixed (an always-false @if), partial (fixed + an unfixable warning after the removed block), rejected (the reverify gate refuses it), cap (1,001 unused keys), cap-fixable (1,001 empty @if blocks), analysis-fail (unterminated @if), limit (10 MiB + 1, generated at runtime), config-fail (mds.json = `{`), write-fail (0o555 dir, unix; skipped with a reason where a read-only dir still accepts files) and partial-module (`_p.mds`). Cells: 520 on unix (260 stdin + 260 file), 480 on Windows (240 + 240; no write-fail cells). One test per (input, fixture) group. Normalization, on the recorded streams only: every spelling of the cell's temp dir (as created, canonical, with/without /private, with/without a verbatim prefix, doubled or forward-slash separators) -> `$TMP`, counted in the golden; `.mds-tmp-` + six alphanumerics + `.tmp` -> `.mds-tmp-RANDOM.tmp`, counted; on Windows only, separators -> `/` (an escaped doubled backslash first). No sorting, trimming or line-ending rewrite. The child runs with a cleared environment plus PATH (and SystemRoot/TEMP on Windows) and NO_COLOR=1, piped streams, relative arguments. Fixture dirs live under /tmp on unix so a leaked absolute path is never wrapped by the renderer at machine-dependent positions. Path-leak baseline: 4 `$TMP` replacements (and 4 temp-name replacements) over all 520 cells, all in the four file-mode `--fix` write-fail cells. Storage: 126 deduplicated goldens plus a cell -> golden table, generated into tests/lint_golden/single.rs as single-line literals (Debug-escaped, braced escapes only) via `MDS_GOLDEN_PRINT=1 cargo nextest run -p mds-cli --test lint_golden golden_print_mode --no-capture`, which prints and never writes files. Streams over 16 KiB are stored as a digest: length, line count, FNV-1a-64, the first 40 lines (at most 4 KiB) and the last 20 lines (at most 2 KiB). Golden data: 125,778 bytes (capped at 1 MiB by a self-check). Self-checks: literal cell counts per platform; every cell maps to exactly one golden, no golden is unreferenced or duplicated; storage bounds, no control character but newline and no backslash in any golden; a comparator that detects a one-byte change in exit, stdout, stderr, file bytes and both replacement counts (inline and digest) and reports only the cell id, lengths and the first differing line with three lines of context; the normalizer's tempdir and temp-name rules; fixture markers checked against the tool rather than the goldens. Co-Authored-By: Claude <noreply@anthropic.com>
Add the directory cells of the lint characterization matrix: `mds lint ... d` (relative, cwd = the fixture's parent) x human/json x --quiet off/on x the five fix modes x 16 directory fixtures: the 13 outcome fixtures placed inside d/, plus an empty tree, an all-excluded tree (d/node_modules/a.mds only) and a mixed tree. 320 cells on unix, 300 on Windows (write-fail is unix-only); with the stdin and file cells the whole matrix is 840 / 780. Each directory cell runs twice, creating the fixture files in opposite orders, and the two normalized runs must be byte-identical before the result is compared with its golden. A directory golden records the exit code, exact stdout, exact stderr and the state of every file under the fixture directory (unchanged, new bytes, or missing), sorted by relative path. Normalization, storage, digests and the generator are shared with the single-file cells; single.rs is unchanged, and regenerating it still reproduces it byte for byte. Fixtures: - mixed: files with every outcome (clean, warning, fixable error, partial fix, unterminated @if, one byte over the size limit), a nested mds.json raising unused-variable to an error, a nested malformed mds.json, names whose byte order differs from path-component and case-insensitive order (B.mds, _unterminated.mds, api-utils.mds, api/x.mds), a hidden directory, node_modules and a non-.mds file. - write-fail, directory form: d/ at 0o555 holding the fixable x.mds, unreadable.mds at 0o000, and unreadable/ at 0o000 holding y.mds. It is the only unix-only group, so the chmod-000 cases live here to keep the planned counts. The permissions are probed after they are set; when they do not hold (euid 0, a filesystem that ignores mode bits) the cell is skipped with a printed reason. Goldens: 108 deduplicated directory goldens holding 128,293 bytes of golden text; 254,071 bytes across both tables (cap 1 MiB). Temp-directory replacements over the directory cells: 4, temp-name replacements: 4, all in dir/{human,json}/{loud,quiet}/fix/write-fail. That is the directory-mode path-leak baseline. Today's output is pinned deliberately, including output that is known to be wrong: an over-size file exits 2 and is counted "with errors" in human directory mode (3 and "resource-limited" in JSON); a directory-mode write failure prints the path twice and leaks the absolute temp path; empty and all-excluded trees emit no JSON envelope; a .mds file inside an unlistable directory is skipped without a word (#437). Self-checks: literal counts (840 / 780 by cfg, and per input), exactly one golden per cell across both tables, no dead or duplicate golden, fixture markers checked against the tool (positive controls for the skipped entries and the permission layout, at most one @define per fixture), storage bounds over both tables, and one-byte mutation detection for the directory comparator and the double-run check. The no-backslash rule now covers every golden that a cell running on Windows maps to; a golden that only unix-only cells map to may hold a backslash only as JSON's escaped quote, because the directory JSON write-failure message quotes the temp path. Co-Authored-By: Claude <noreply@anthropic.com>
Reduce duplication between the single-file/stdin and directory runners in lint_golden/harness.rs, with no behavior change: - Cell::id/DirCell::id and Cell::args/DirCell::args shared their "format/quiet/fix-mode/fixture" id suffix and their "lint" + format/quiet/fix flags via new variant_id and lint_args helpers. - group_cells/dir_group_cells shared their (format, quiet, fix mode) cartesian product via a new variant_combinations helper. - compare/compare_dir shared the ExpectedOutputs/ObservedOutputs construction from Golden/DirGolden and Observed/DirObserved via new expected_outputs/dir_expected_outputs/observed_outputs/ dir_observed_outputs helpers, following this file's existing xxx/dir_xxx naming convention (golden_data_bytes/ dir_golden_data_bytes, golden_strings/dir_golden_strings, etc.). Generated data files (single.rs, dir.rs) are untouched and confirmed byte-identical to a fresh MDS_GOLDEN_PRINT=1 regeneration. Co-Authored-By: Claude <noreply@anthropic.com>
Two ways the characterization goldens could mislead, closed without changing any golden (single.rs and dir.rs are untouched; no cell id changes): - An mds.json above the fixture directory applied to every cell. mds lint walks every ancestor for the nearest mds.json, and on unix the fixture directories live under the shared, world-writable /tmp, so a stray /tmp/mds.json turned every cell into a golden diff with no hint at the cause (with one raising unused-variable to error: "20 of 20 cells differ ... exit: expected 1, actual 2"), and during generation it would have been baked into the goldens. fixture_dir() now stops with the offending path; ancestor_config() does the lookup on the canonical path, as the product does, and a new test plants an mds.json one level up as its positive control. - A skipped cell passed silently. When a permission precondition does not hold (euid 0, or a filesystem that ignores mode bits) the 60 write-fail cells and the write-fail markers were skipped with a line on captured stderr only, so a green run hid their loss. Skips are now announced as a SKIPPED line and, under GitHub Actions, a job-summary warning with the skipped/total count (the cli_watch.rs convention), and a group in which no cell ran or was skipped fails instead of passing with nothing compared. Co-Authored-By: Claude <noreply@anthropic.com>
The watch suite waited on stderr with a helper that returned whatever it had when the time ran out, so a line that never appeared was reported by a later assertion as if it were a final answer, and several counts and absences were read from a snapshot of a live pipe. tests/common/mod.rs: - wait_for_tap and wait_for_tap_count panic on timeout; #[track_caller] and the message opens with the caller's file:line:column and ends with what the tap held. wait_for_tap_count is the former cli_watch.rs wait_for_stderr_count, moved and renamed (it polls any pipe tap). - poll_tap_until: the non-panicking poll both are built on, for the cap test, which treats "no rebuild yet" as data. - ORDER_MARKER_SOURCE / ORDER_MARKER_LINE: a source whose compile fails with one diagnostic. The watch loop finishes a rebuild before it takes the next event, so once the marker's diagnostic is on the tap every earlier rebuild's lines are too: an ordered anchor for exact counts. - tap_reader is public so the waits can be tested without a process; ChildGuard::wait_status is #[track_caller]. tests/cli_watch.rs: - wait_for_stderr_contains_str is gone; its 21 call sites use wait_for_tap, except the cap test, which uses poll_tap_until. - Every wait_* helper is #[track_caller]. - I8 and I9 read their exact final counts with finish_text behind the order marker; I18's two zero counts wait for the Compiled to / Recompiled line that follows the warning; I20's zero waits for the marker, since --quiet silences every status line. The burst test and the single-status-line test take their exact counts the same way. - The 500 ms sleeps: the three that stood in for an event are waits on it (quiet error, compile error, missing partial at startup); the two first windows of the error-settle tests now start from the first error they measure; the remaining four are negative windows, commented, each with a later positive anchor. - Self-tests catch the waits' panics with catch_unwind (no panic hook) and assert the message names this file's line of the call. Mutation controls, not committed: an extra duplicate-warning emit at the rebuild sites of watch.rs fails I8, I9, I18 and I20 at their count assertions; one at the file-mode startup site fails I18's startup zero and I20; dropping #[track_caller] from the waits fails both panic self-tests. Product source is unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
When every attempt of the watch cap test is inconclusive it skips by returning early. cargo test counts that as a pass and never shows a passing test's stderr, so the skip was invisible in CI logs and the watch soak tallied it as a pass. tests/cli_watch.rs: - The writer thread records when each write completed; every attempt logs a gap breakdown on stderr: write count and span, max / median / p99 gap, gaps at or over the window, and when the first rebuild was seen with the largest gap before it. The precondition still judges the same metric, the largest gap measured from the writer's start, so the test behaves as before. - A skip goes through record_skip: the SKIPPED line on stderr (kept), one line appended to the file MDS_TEST_SKIP_LOG names (a set variable that cannot be written fails the test, since whoever set it is counting), and the job-summary warning under GitHub Actions. The line carries every attempt's largest gap and the last attempt's breakdown. .github/workflows/watch-soak.yml (still workflow_dispatch only; no new job, both legs keep identical bounds): - A `test` input, allowlisted to cli_watch, cli_watch_cap and cli_watch_truncate and validated beside `iterations`, replaces the hard-coded `--test cli_watch`; a value outside the allowlist, or a test file absent at the dispatched ref, fails before the build. - MDS_TEST_SKIP_LOG is set per iteration; an iteration that passed but logged a skip is SKIP, tallied apart from pass and fail in summary.txt and the step summary, with its log kept in the artifact. - The tally lives in one shell helper sourced by the soak and by a new self-check step, which plants one skip line and asserts it counts as exactly one skipped iteration before anything is built. Checked locally: the three new run blocks, extracted verbatim, pass bash -n; the self-check passes and fails when the helper counts a skip as a pass; the loop, driven by a fake cargo, reports PASS, SKIP, FAIL, PASS with both summaries; the validation accepts cli_watch and rejects cli_lint, an empty value and a value with a newline (printed on one line). A temporary change that forced the skip path appended exactly one line to MDS_TEST_SKIP_LOG, which the tally counted as one SKIP. Co-Authored-By: Claude <noreply@anthropic.com>
The earlier #381 anchors left four count checks, and one negative window, unable to see the failure they guard; seven other zero-count or absence checks read the stream the same unordered way. - I16, I17 and I19 stopped the watcher the moment the expected count was reached, and I18's final count stopped at its second Recompiled line: a surplus warning printed just after was cut off by the kill. Each final count is now read behind the order marker and gains a Recompiled control. I16's startup and first-rebuild counts are anchored on the line file mode prints after them (Compiled to, Recompiled). - watch_import_removal_stops_tracking_dep could not observe the rebuild it forbids: the entry rendered the same text either way, and a rebuild that changes nothing is silent. Once it drops the import, the entry now @includes an empty module, so every compile of it warns; the test runs without -q and without the idle tick, and counts that warning across the helper edit behind the marker. - R3 read its absences from a live snapshot. The three startup no-spurious-rebuild tests, both idle tests and the 500-file idle test read a zero after a fixed window, with nothing proving the loop was still live. Each now writes the order marker after its window and reads behind it. - common: the order marker's line can print several times for one edit; wait for it, never count it. - watch-soak.yml: a rejected iterations value is printed with %q, as a rejected test value already was. Echoed raw, a value holding a newline started a line the runner reads as a workflow command. Mutation controls (product source restored and identical to bff9843 after each): a delayed second vars-file warning after every rebuild fails the new I16, I17, I18 and I19 at their final counts, while the previous I17, I18 and I19 passed; keeping a removed import tracked fails the import-removal test (previous passed); a late .tmp- status line fails R3 (previous passed); a loop that stalls after readiness fails all six window tests at the marker wait (previous passed). Co-Authored-By: Claude <noreply@anthropic.com>
… exit funnel `eprintln!` panics when stderr is gone, so a run whose stderr reader had exited ended with exit 101 whatever its verdict. This adds the machinery the #157 conversions build on, and moves main.rs and the error and warning helpers onto it: - output.rs: `ewrite!` / `ewriteln!` over `write_stderr_fmt`, which never panics. A closed pipe records a sticky stderr-closed fact and drops every later stderr write; any other failure records a sticky I/O-failed fact. The facts live in `OutputState` (one atomic); everything that decides from them takes `&OutputState`, so unit tests build their own. - `StdoutOutcome { Written, Closed, Failed }` and `write_stdout`, which writes and flushes and is Written only when both succeed. Its callers arrive with the build and fmt conversion. - `final_exit_code(verdict, &OutputState, ExitPolicy)`: a closed pipe never changes the code; under Batch any other output failure lifts it to max(verdict, 2); WatchSession keeps the verdict. `exit(verdict)` is the funnel, and `main` now ends in it. - main.rs: its 8 stderr prints and 6 `process::exit` calls go through the writer and the funnel. - `eprint_error` and `eprint_warning` write through `ewriteln!`. - `render_error_sanitized` no longer panics when a Display fails. The sanitized wrapper renders every text surface (code and url now included) without `to_string()`: a failing optional surface is dropped, a failing message becomes a placeholder. The frame renders through `fmt::Write`, and a render that fails part-way falls back to escaped plain text. - print_discipline.rs lists `ewriteln!` / `ewrite!` with the print macros, so every converted site stays under the guard, and adds per-file site floors for main.rs (11) and output.rs (7): with the macros unlisted, the crate-wide floor still passed (98 >= 80) while ten sites had left it. Tests: tests/broken_pipe.rs runs `check` (file, broken file, stdin, directory), `init` and `lint --format yaml` with stderr as a pipe whose reader is dropped before spawn, each beside an open-stderr control, and asserts the exit code, stdout and the files written. Unit tests pin the writer's sticky states through an injected sink, the stdout outcome mapping, a final_exit_code table for both policies, and the renderer fallback. RED at the parent: all six broken_pipe rows exited Some(101) with stderr closed while their open-stderr controls passed; the renderer tests panicked in to_string ("a Display implementation returned an error unexpectedly") and in format! ("a formatting trait implementation returned an error when the underlying stream did not"). Co-Authored-By: Claude <noreply@anthropic.com>
print_discipline scans the stderr writer macros by name, but not the
function they expand to. `write_stderr_fmt` is crate-visible and takes a
`format_args!`, which is not a print site, so a direct call wrote to
stderr with nothing it interpolated scanned, and so would every site of a
new macro built on it under a name the guard does not list. A planted
`output::write_stderr_fmt(format_args!("RED {}\n", dir.display()))` in
run_check_directory passed all 12 print_discipline tests: the per-file
floors only see a site leave a floored file, and main.rs kept its 11.
- print_discipline.rs: `the_stderr_writer_fn_is_called_only_by_the_writer_macros`
fails on any mention of `write_stderr_fmt` outside its `fn` definition
and the bodies of the `macro_rules!` definitions PRINT_MACROS names, a
renamed `use` of it included, and asserts it found the definition and
the writer macros' bodies in output.rs.
`the_writer_fn_guard_flags_direct_calls_aliases_and_unlisted_macros`
pins the accepted shapes, the reported ones (a direct call, a renamed
import, an unlisted macro built on it) and that comments, strings and
longer identifiers are not mentions. `matching_delim` extends
`matching_paren` to `[]` and `{}` for the macro bodies. The module doc
lists the new coverage and records renaming a print macro on import as
an accepted limit, the same shape as a renamed sanitizer.
- output.rs: the module doc names the writer macros as the stderr choke
point and eprint_error as the error-report choke point; two test
comments said eprint_warning calls eprintln!, where it calls ewriteln!.
Controls: with the planted call the new guard fails naming main.rs:396
while the 12 existing tests pass; accepting every macro body fails the
self-test (stray [3, 5] instead of [3, 5, 8]); a writer name that does
not exist fails the non-vacuity check (found 0). No product behaviour
changes; the lint goldens are unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
…mmary GitHub starts every run: block as `bash --noprofile --norc -eo pipefail`, so the soak loop's plain `cargo test` call followed by a read of `$?` ended the whole soak at the first failing iteration, before summary.txt and the step summary were written; the loop's own `set -uo pipefail` cannot clear -e. The loop body now lives in one sourced file, captures each exit code with `|| rc=$?`, runs every iteration, writes both summaries, and fails the step only at the end. The self-check now drives that real loop under the runner's exact flags with a planted cargo stand-in (pass, fail, skip, pass, zero tests) and fails against the old loop shape. The `filter` input is validated to A-Z, a-z, 0-9, _ and :, rejected values are printed with %q, and an iteration that runs no test (a filter matching nothing, or no test count at all) counts as a failure. The `iterations` check rejects over-long values, which `[` could not compare and which ran zero iterations. Proven live on a throwaway ref before this commit: smoke passed, a `filter=--list` dispatch failed at input validation, and a 50-iteration soak passed on both legs. Co-Authored-By: Claude <noreply@anthropic.com>
mds build and mds fmt printed through std's eprintln!/print!, which panic when the write fails: a closed stderr or a closed stdout ended the run with exit 101, and a directory build stopped at its first status line, so the files after it were never written. - build.rs and fmt.rs print every status line through output::ewriteln! and end the run through output::exit. - write_output's stdout arm writes through output::write_stdout. A closed stdout is not an error: nothing more is written and the verdict stands. Any other stdout failure is mds::io. - fmt's own write_stdout/print_diff are gone; its filter output and diffs go through output::write_stdout the same way. - StdoutOutcome::into_batch_result maps an outcome for a run that ends on its own. It records nothing; the error reaches the exit code through the caller. Tests: - broken_pipe.rs closes stdout or stderr per row. New rows: build to stdout from a file and from stdin (stdout closed), build a file, with a source map, a directory that compiles and one with a broken file (stderr closed, every output still written), fmt a file and fmt --check (stderr closed), fmt stdin, --diff and --check --diff on a file and a directory (stdout closed). The build and fmt stderr rows and both build stdout rows exited 101 before this change. The fmt stdout rows passed already: fmt's old helper treated a closed pipe as success, and they now guard its removal. - Two Linux-only rows put stdout on /dev/full: build -o - and fmt - must exit 2 with one mds::io error naming stdout. They are ignored off Linux. - cli_fmt.rs: the closed-pipe test drops the reader before the spawn, asserts exit 0 and compares against an open-pipe run that receives the formatted text. - print_discipline.rs: per-file floors for build.rs (30) and fmt.rs (11); the allowlist entry for build.rs's old print! of the compiled output is deleted. mds watch -o - shares write_output: when its stdout reader goes away it now keeps watching instead of panicking. Stopping the session there is the watch half of #157. Lint goldens are unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
A failed write, a failed mkdir and an unreadable stdin were uncoded
miette! errors, so build, check, fmt and init exited 1 for them - the
template-error code. A stale sibling that could not be removed only
warned (exit 0), and stdin over the 10 MiB cap exited 1 (2 under lint)
where a file over the same cap exits 3.
- atomic_write_file returns MdsError::Io (mds::io, exit 2) with its
wording unchanged; so do an output directory that cannot be created, a
stale sidecar or sibling that cannot be removed (now an error), and a
stdout write that fails for another reason than a closed pipe.
- read_stdin: over the cap is mds::resource_limit (exit 3); a read that
fails and bytes that are not UTF-8 are mds::io (exit 2). lint's stdin
path exits with the code's class (its human error is unchanged).
- output::note_io_failure records an I/O failure for the exit funnel, and
output::eprint_io_failure reports and records it. Directory builds and
fmt <dir> use it and carry on with the other files, exiting 2; a
template error alone still exits 1. write_output records nothing.
- main parses with Cli::try_parse and prints clap's help, version or
usage error under the same rule: a closed pipe changes nothing, a
failed stdout write is mds::io and exits 2.
- watch keeps its stale-sibling warning text (a rebuild's failure does
not change how a session exits); lint.rs and watch.rs change only
where atomic_write_file / read_stdin / probe_and_remove_stale changed.
Tests updated from exit 1 to exit 2 on a write failure:
- cli_commands.rs init_force_symlink_target_refused_plain_path_created
- cli_commands.rs init_dangling_symlink_refused_without_force
- cli_build.rs build_refuses_every_output_route_that_lands_on_the_entry
(symlink at the -o path)
Strengthened from "not 0" to exit 2 and mds::io:
- cli_build.rs build_o_write_failure_preserves_existing_output_no_temp_residue
- dir_build.rs dir_build_write_failure_preserves_existing_outputs
- dir_build.rs dir_build_source_map_sidecar_symlink_target_rejected
- security.rs stdin_size_limit_rejects_oversized_input (now exit 3)
- build.rs unit read_stdin_holds_at_most_the_cap_plus_one_byte
New tests: broken_pipe.rs rows for an unwritable output (build file,
build dir, fmt file, fmt dir, lint --fix, init), stdin that cannot be
read (a directory handle on unix, a write-only handle on Windows), stdin
that is not UTF-8, stdin over the cap (build, check, fmt, lint, lint
--format json) and exactly at the cap, clap's --help and --version with
stdout closed and a usage error with stderr closed, and --help into
/dev/full (Linux only). dir_build.rs: a template error plus a failed
mkdir exits 2 (a template error alone exits 1), and a stale sibling that
cannot be removed exits 2. build.rs unit: a stale map that cannot be
removed is mds::io.
print_discipline.rs floors: build.rs 30 -> 27 (four prints became
mds::io reports; one unit-test skip line counts), output.rs 7 -> 6 (the
stale-sibling warning moved to its watch caller).
Lint goldens changed only in these cells (goldens 49, 120, 121 of
single.rs; dir.rs byte-identical):
- file/human/loud/fix/write-fail, file/human/quiet/fix/write-fail,
file/json/loud/fix/write-fail, file/json/quiet/fix/write-fail
(stderr gains the mds::io code line)
- stdin/human/{loud,quiet}/{report,fix,fix-check,fix-diff,fix-check-diff}/limit
and stdin/json/{loud,quiet}/report/limit (exit 2 -> 3, stderr now
mds::resource_limit)
Docs: spec.md section 5 (a "Streams and I/O failures" rule, the mds::io
and mds::resource_limit rows), 7.2 (continue-on-error, stale-flip
cleanup, --quiet, output writing), 7.4, 7.6 and 7.9; README exit codes;
CHANGELOG BREAKING (CLI) entry. The build.rs "documented limitation"
comment is gone: the stale-unlink failure is now an error.
Co-Authored-By: Claude <noreply@anthropic.com>
exit_after_clap_output's match on the print-and-flush result used a
guarded Err arm plus a catch-all `Ok(()) | Err(_) => {}` to do nothing on
success and on a closed pipe. An `if let Err(e) = printed { if e.kind()
!= BrokenPipe { ... } }` says the same thing without the empty arm.
Reviewed the rest of the #157 stream-handling diff (output.rs, build.rs,
fmt.rs, and the broken_pipe/cli_build/cli_commands/cli_fmt/dir_build/
security test files) for slop, duplication and unclear naming; found
none worth changing. Behaviour, exit codes, and message wording are
unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
…tdout stays closed A directory build, check or fmt reported a source it could not read as mds::io but exited 1, while the same file given alone exits 2, and spec sections 5 and 7.9, the README and the fmt module doc promise 2. Each per-file failure now goes through output::eprint_file_failure, which records an I/O failure for the exit code when the error is in the exit-2 class (mds::io, mds::file_not_found, mds::not_mds) and then reports it. A directory run therefore exits 2 for a source that cannot be read, a path carrying a forbidden character (#265) or an @import of a file that does not exist, as that file alone does; a run whose failures are only template errors or resource limits still exits 1. Four walker tests in forbidden_paths.rs and one in security.rs pinned exit 1 for the forbidden-character refusal in directory mode; they now expect 2. write_stdout now remembers a closed stdout: once a write hits a closed pipe, OutputState records it and every later write returns Closed without writing. A stdout that fails for another reason is recorded too, and a repeat of that failure returns the new StdoutOutcome::FailedAgain, which a batch run does not report again: mds fmt --diff <dir> into a full device prints one mds::io frame, not one per file, and counts every file whose diff was lost as failed. A stale .map sidecar that could not be read (its metadata, open or read failing for a reason other than not-found) was reported as "not a tool-generated SMv3 map" and left in place with exit 0. It is now mds::io, "cannot read stale map <path>: <cause>", exit 2. The oversized-stdin message names the cap it enforces, 10 MiB, instead of 10 MB. Golden 49 of lint_golden/single.rs changed (message text only); its cells: stdin/human/loud/report/limit, stdin/human/loud/fix/limit, stdin/human/loud/fix-check/limit, stdin/human/loud/fix-diff/limit, stdin/human/loud/fix-check-diff/limit, stdin/human/quiet/report/limit, stdin/human/quiet/fix/limit, stdin/human/quiet/fix-check/limit, stdin/human/quiet/fix-diff/limit, stdin/human/quiet/fix-check-diff/limit, stdin/json/loud/report/limit, stdin/json/quiet/report/limit. lint_golden/dir.rs regenerated byte-identical. The CHANGELOG #157 entry no longer says a closed stdout used to panic under mds fmt (fmt already tolerated it), names the directory-mode change, and scopes its "every other I/O failure" to the failures it lists, as spec sections 5, 7.2, 7.4 and 7.9 and the README now do: an unreadable mds.json stays an uncoded exit 1. The closed_pipe() test helper moves to tests/common/mod.rs, shared by broken_pipe.rs and cli_fmt.rs. Co-Authored-By: Claude <noreply@anthropic.com>
…iled stdout write mds lint printed its status lines, its directory summary and its own usage errors with eprintln!, so a closed stderr ended a lint run with a Rust panic and exit 101: `mds lint ok.mds` (Clean:), `mds lint <dir>` (the summary), `mds lint --fix` (Fixed:), `mds lint --fix --check -` (Would fix:). Its local write_stdout treated a closed pipe as success, but every caller dropped any other stdout failure, so a clean `mds lint --format json` into a failing stdout exited 0 with its report lost. Every stderr print in lint.rs now goes through the non-panicking stderr writer, and every exit through the exit funnel. The local write_stdout is gone: the JSON report, the --diff output and the fixed source are written with output::write_stdout, and lint handles the outcome in one place. A closed stdout keeps the verdict. The first other failure is reported as one mds::io error naming stdout and recorded, so the funnel lifts the exit code to at least 2; a repeat is not reported again, and a resource limit keeps its 3. broken_pipe.rs gains the lint rows: stderr closed (a clean file, a file, a directory and stdin with warnings, --fix, --fix --check -) and stdout closed (--format json on a file, a directory and stdin, --fix -, --fix --diff) keep their verdicts; on Linux, stdout on /dev/full makes a clean JSON report of a file or a directory and --fix - exit 2, keeps 3 for a file over the size cap, and reports two lost diffs once. The lint goldens are unchanged. README and the #157 CHANGELOG entry now cover mds lint. Co-Authored-By: Claude <noreply@anthropic.com>
mds fmt --diff <dir> counts a file whose diff a stdout failing other than by a closed pipe lost as failed. mds lint --fix --diff <dir> reported the same failure once and lifted the exit code, but left the file in its findings bucket, so its summary read "2 clean, 0 with warnings, 0 with errors, 0 resource-limited" above exit 2. Decision: one rule for both, the one every directory run already applies to a file whose output fails. A file whose own diff a failing stdout lost counts as failed: "N failed" under fmt, "with errors" under lint, which is where lint already counts a --fix rewrite that fails. The failure is still reported once for the run. A closed stdout loses nothing and moves no file. Run-level writes (lint's JSON envelope, the single-file and stdin output) belong to no one file; the recorded failure lifts their exit code as before. fmt already followed the rule and is unchanged. lint's emit_stdout now says whether a failing stdout lost its text, and both directory emitters, human and JSON, count such a file under "with errors". broken_pipe.rs: every unix now has a stdout failure that is not a closed pipe, a regular file the child may not grow (file-size limit 0, SIGXFSZ ignored, so writes fail with EFBIG), beside Linux's /dev/full. The lint failing-stdout rows run on it, so they run on macOS too, and were renamed from "into a full device" to "into a failing stdout". New rows pin the rule for fmt and for both lint directory emitters, and that a closed stdout moves no lint directory file, for both emitters. The fmt /dev/full directory row became the fmt case of the shared rule. spec: the "Streams and I/O failures on the CLI" block names mds lint, the directory counting rule, and the one exception to mds::io (a directory rewrite failure under --format human prints "error writing"); lint's "with errors" bucket and its exit-2 row list these I/O failures. README dates lint's old exit 0 and states the counting rule; the #157 CHANGELOG entry scopes lint's mds::io rewrite report to a file argument and adds the counting rule. The lint goldens are unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
…- loses its reader mds watch printed its status lines with std's eprint macros, which panic when the write fails: a closed stderr ended the session with exit 101 at its first line, before it was watching. Every status line now goes through the CLI's stderr writer, which never panics; with stderr closed the session reaches readiness, rebuilds on every edit, and exits 0 at Ctrl+C. `ewrite!` gets its first caller (the --clear sequence). With -o -, stdout is the session's product. A write that finds its reader gone (a closed pipe) now ends the session: one `Stopped watching (stdout closed).` line, suppressed by --quiet, and exit 0, at startup or on a rebuild. Since the stream writer landed, such a session kept watching and printed `Recompiled <stdout>` for writes that went nowhere. The session reads each stdout outcome itself instead of through the batch reading, which takes a closed pipe and a repeated failure for success: - the first stdout failure for another reason is reported once as mds::io naming stdout, and watching continues; - a repeat of it is not reported again, is not announced as `Recompiled`, and is not recorded as written, so the content-dedup baseline keeps what was last written and a save of the same text writes again. Exit policy: a watch session that goes live (every watch armed, every baseline captured) records it in the output state, and the exit funnel then applies the watch-session rule, so an output failure during the session (a failed rebuild write, a stdout or stderr that fails other than by a closed pipe) never changes the Ctrl+C exit. A session that ends before going live keeps the batch rule. The go-live point is one function, shared by file and directory mode, which switches the policy before it writes the readiness marker. The output state stays private to output.rs. Tests (cli_watch.rs): stdout's reader gone before the spawn stops the session with the line and exit 0, silent under --quiet (open-pipe control); a reader that goes away after the first output stops it at the next rebuild; stderr closed from the start keeps watching in file and directory mode and exits 0 at Ctrl+C (open-stderr control); a stdout file the child may not grow is reported once, a repeat claims no rebuild, and a same-text save after room is made writes the output; a rebuild write that fails and a stderr that fails other than by a closed pipe leave the Ctrl+C exit at 0 (control: that stderr lifts mds check to 2). Unit tests pin the session's reading of each stdout outcome and the policy switch. The file-size-limit helper moves to tests/common so cli_watch and broken_pipe share it; the stream file is opened at its end rather than for appending, since macOS checks the limit against the descriptor offset. print_discipline gains a watch.rs site floor (18). README and the CHANGELOG #157 entry describe the watch behaviour. Co-Authored-By: Claude <noreply@anthropic.com>
…writer and the funnel print_discipline gains the two lexical bans the stream rule rests on. no_raw_print_macro_outside_the_writer: println!, print!, eprintln!, eprint! and dbg! panic when their write fails, which is how a closed or failing stream ended a run with exit 101. None may be named anywhere in crates/mds-cli/src: not called (path-qualified included), not renamed on import, not wrapped in another macro. There is no exemption, neither for test modules nor for the writer macros' own bodies: the writer expands to write_stderr_fmt and needs none, and a writer body that reached for eprintln! would bring the panic back. A method of that name (clap's err.print()), a function definition and a longer identifier are not mentions; clap's own output stays outside the scan by construction, as the guard doc now says. process_exit_only_in_the_funnel: std::process::exit may appear only in the body of output::exit, the funnel that applies the output-failure rule (EXIT_FUNNELS lists it; a second funnel is added there, never exempted elsewhere). std::process::abort appears nowhere, and a use item that would let a call avoid naming process::exit (renamed, braced, globbed, or a renamed process module) is reported. Both bans read every module the crate compiles: the source list is checked against each `mod name;` resolved from main.rs, and a #[path] module fails closed. every_printing_file_has_a_site_floor requires a SITE_FLOORS entry or a written reason in FLOORS_PENDING for each file that prints; lint.rs is the one pending file, until the lint pipeline refactor (#309) settles its site count. Synthetic positive and negative controls cover each scanner. Product-side change, test code only: the ban has no test-module exemption, so the four test-only skip notices (two in build.rs's tests, two in output.rs's tests) move from eprintln! to the writer macro. They print only when a test cannot run its vector (running as root, or Windows without symlink rights), and they now reach the real stderr uncaptured. No production code changes. broken_pipe.rs: the subcommand x closed-stream table is complete. The two missing cells are commands that write nothing to stdout, check and init; their rows assert the open run's stdout is empty and that closing it changes nothing. TABLE names one row per subcommand and stream (the watch rows live in cli_watch.rs), and every_subcommand_has_a_row_for_each_closed_stream fails for a subcommand `mds --help` lists without both rows, a row whose subcommand is gone, and a row that names no #[test] function. Co-Authored-By: Claude <noreply@anthropic.com>
stop_watching's StopReason match had one braced arm and one bare expression arm for what are both single ewriteln! calls; make both arms the same shape. Co-Authored-By: Claude <noreply@anthropic.com>
spec.md described no watch stream behaviour. Section 5's "Streams and I/O
failures on the CLI" now has a paragraph for `mds watch`: a closed stderr
keeps it watching; with `-o -` a gone reader ends the session with
`Stopped watching (stdout closed).` (not under --quiet) and exit 0; another
stdout failure is reported once and not recorded as written, so a save of
unchanged content retries it; once the session is live no output failure
changes the Ctrl+C exit; a session that ends before it is live follows the
batch rule.
Two of those promises had no test:
- the batch rule before the session is live. A new unix test stops a
session at startup on a gone stdout reader while stderr fails with EFBIG:
exit 2, with an open-stderr control that exits 0. Marking the session
live at the top of file mode makes it fail (exit 0).
- `--quiet` for a reader that goes away mid-session. The mid-session test
now runs a loud and a quiet arm. Passing `false` for quiet at the file
loop's stop line makes the quiet arm fail.
The print-discipline guards had gaps:
- the exit ban missed a `process` renamed through a braced `self`
(`use std::process::{self as p, Command};`) and a rename after another
`process` path in the same braces. Both are now reported. The synthetic
control failed on both lines before the scanner change.
- nothing kept a second writer from holding its own stdout or stderr
handle, which would skip the closed-pipe and report-once state, as lint's
duplicate `write_stdout` did. A new ban allows `stdout`/`stderr` only in
`write_stdout`, `write_stderr_fmt` and main's exit after clap's output,
or for an `.is_terminal()` query. It has synthetic controls, a check that
each owner holds its handle, and a planted handle in watch.rs makes it
fail.
watch.rs doc comments that said every stop exits 0 now name the startup
exception.
Co-Authored-By: Claude <noreply@anthropic.com>
…recovers A stdout write that failed for a reason other than a closed pipe was reported once for the whole process: the stdout-failed fact was sticky, so every later failure came back as a silent repeat. For a batch run that is the intended report-once. For a long `mds watch -o -` session it hid failures: stdout failed (reported), recovered (a rebuild wrote and printed `Recompiled`), then failed again with no report and no `Recompiled`. Now the report is once per failure: a stdout write that lands clears the stdout-failed fact, so the next failure is `Failed` again and reported. A write of no bytes proves nothing about stdout and clears nothing. A closed stdout stays closed. The fact that lifts a batch run's exit code to 2 is a separate bit and is never cleared, so a batch run that lost a stdout write still exits 2 after stdout recovers; a batch stdout that keeps failing is reported once, as before. Tests: a unit test drives the writer through fail, repeat, empty write, recovery, fail, repeat and checks the exit-lifting fact survives; a unix watch test makes stdout fail at startup, recover for one rebuild and fail again, and expects two reports, one `Recompiled` and exit 0 at Ctrl+C. Docs: spec §5 (the batch list, the watch paragraph) and the `mds fmt` directory sentence say "once while it keeps failing"; README and the #157 CHANGELOG entry say the same, and now state the exception spec already had: a session that stops at the startup write while stderr has also failed other than by a closed pipe exits 2, as the other commands do. Lint goldens unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
`LintSource` in lint.rs is the one place a lint input gets its names:
`Stdin`, `File { typed, name }` and `DirEntry { path, key }`, with
`display_label()` (the name its diagnostics carry: each diag.file, the
JSON files[].file key, the name a human frame renders the source under)
and `diff_label()` (the escaped name a --fix --diff header shows).
What moved:
- stdin: 7 of the 9 STDIN_DISPLAY_LABEL sites now go through
`LintSource::Stdin`: the diagnostic relabel, the preview and apply
labels, the three named sources and the diff header. The `Would fix:`
and `Partially fixed:` status lines keep the constant inside the
writer macro: print_discipline accepts only an escape call or an
allowlisted name there, and its allowlist liveness check fails once
lint.rs stops naming STDIN_DISPLAY_LABEL in a macro argument (planted
and observed failing).
- single file: the `<file>` fallback label is gone. `LintSource::file`
parses the typed path into its UTF-8 file name once, after the read;
the relabel, the named source, the preview and apply labels, both
`Clean:` lines and the diff header use it.
- directory: both per-file functions build a `DirEntry` from the walked
path and its root-relative key; the relabel, the error-entry JSON key,
the named source, the preview and apply labels and the diff header use
it.
- Status lines that name a path keep `safe_path(..)` inside the macro.
The `<file>` fallback was dead. `run_lint_file` has one caller, reached
only after `ensure_existing_mds_file` accepted the path, so its
extension is `Some("mds")` and `file_name()` is `Some` (the extension is
read from it). The fallback was evaluated after `read_source_file`
succeeded: `NativeFs::check_symlink` refuses a final-component symlink
and canonicalizes `canonicalize(parent).join(file_name)`, requiring the
result's parent to be that canonical parent, so the canonical path ends
in the file's own name; `read_canonical_source` then refuses a canonical
path that is not valid UTF-8 ("path is not valid UTF-8", exit 2, pinned
on Linux by cli_lint's lint_single_file_non_utf8_path_exits_2). The file
name was therefore always UTF-8 where the fallback ran. `LintSource::file`
keeps an `mds::io` refusal for that impossible case instead of a label.
No behaviour change: goldens and tests unchanged (crates/mds-cli/tests
is byte-identical to the parent commit; lint_golden 52/52, cli_lint 129,
print_discipline 22, mds-cli nextest 1124 passed + 3 skipped, bin 196).
Co-Authored-By: Claude <noreply@anthropic.com>
PR CI run 36703033927 failed in three jobs on one clippy error under Rust 1.98.0: clippy::byte_char_slices ("can be more succinctly written as a byte str") at crates/mds-cli/tests/print_discipline.rs, where asks_is_terminal loops over [b'(', b')', b'.']. The loop now iterates *b"().", clippy's suggestion; it yields the same three u8 values in the same order, so the guard behaves identically.
The error slipped through because the local default stable toolchain is Rust 1.96.0, whose clippy predates the lint, while CI's stable is 1.98.0. Local fmt and clippy gates now run with +1.98 to match CI.
Co-Authored-By: Claude <noreply@anthropic.com>
`run_lint_file` names its input with `LintSource::file` once `mds::lint` has accepted the path, instead of straight after the read. `mds::lint` refuses a path that is not valid UTF-8 before it reads anything, so once it has returned the typed file name is UTF-8 and the `mds::io` refusal in `LintSource::file` cannot be reached. The read before it does not guarantee that on its own: it validates the canonical path it opens, not the path as typed. In this order every refusal on this path is `mds::lint`'s own, exactly as before `LintSource` existed. The doc of `LintSource::file` now gives this reason. No behaviour change: goldens and tests unchanged (crates/mds-cli/tests is byte-identical to the parent commit). Co-Authored-By: Claude <noreply@anthropic.com>
…arator
Four broken_pipe rows failed on Windows in their open-stream control: the
expected text hard-coded `/` in a path mds builds by joining components,
which Windows prints with `\` (`Compiled to .\ok.md`, `+++ fixes\b.mds`).
The product's output is right on each OS; the expectations were not.
The expected paths are now built with `native()`, which writes a path
spelled with `/` in `std::path::MAIN_SEPARATOR_STR`, and `diff_header()`
(`+++ <path>\n` through `native()`), so each control is still an exact match
on every OS. `Row` and `FullRow` borrow their expected text for a lifetime
instead of `'static`, to hold the built strings.
Changed expectations (unix text unchanged; Windows now `\`):
- build_a_file_with_stderr_closed_exits_0_and_writes_the_output:
"Compiled to ./ok.md\n"
- build_with_a_source_map_and_stderr_closed_exits_0_and_writes_both_files:
"Source map written to ./ok.md.map\n"
- lint_fix_diff_of_a_directory_with_stdout_closed_exits_1 and
lint_json_fix_diff_of_a_directory_with_stdout_closed_exits_1:
"+++ fixes/b.mds\n"
- fmt_check_diff_on_a_directory_with_stdout_closed_exits_1: the path-less
"messy.mds\n" becomes the exact header "+++ m/messy.mds\n"
- the failing-stdout directory rows (unix only today):
"+++ two/a.mds\n", "+++ two/b.mds\n", "+++ fixes/a.mds\n",
"+++ fixes/b.mds\n", so they stay exact if a Windows vector is added
Paths compared as `Path` (`written()`, `files_under()`) and typed paths
printed as given ("+++ fix.mds\n", "Clean: ok.mds\n") need no change.
Co-Authored-By: Claude <noreply@anthropic.com>
`mds lint --fix` and its preview (`--fix --check` / `--fix --diff`) ran
the fix pipeline through two copies: plan_and_apply_fixes returned a
FixFileOutcome, preview_fixes a PreviewOutcome, and each built its own
reverify closure, the two identical in behaviour.
- ReverifyGate replaces the two closures. It is built once per plan
(the original's compiled output, kept only when every planned edit is
output-neutral) and its verify() is what apply_fixes_incremental
calls for each candidate: the candidate must lint, and when the
output check applies it must compile to the same output. The refusal
keeps its wording.
- run_fix_pipeline(&LintSource, ...) replaces both functions, and
FixPipelineOutcome { Fixed, PartiallyFixed, Rejected, NothingToFix }
replaces both outcome enums. The pipeline borrows the lint result
instead of handing it back, so Rejected and NothingToFix carry no
copy of it. `--fix` writes the source of Fixed and PartiallyFixed;
the preview reads either as "would fix", exactly as it read its own
WouldFix before.
- Stdin, single-file and both directory modes call run_fix_pipeline
with their LintSource in place of a display-label string.
In-module tests ported one to one (same fixtures, same assertions):
preview_fixes_surfaces_rejected_on_overlap ->
fix_pipeline_surfaces_rejected_on_overlap, and
preview_fixes_would_fix_relabels_residual_display_path ->
fix_pipeline_would_fix_relabels_residual_display_path. Their labels
come from LintSource::DirEntry values with the same names.
No behaviour change: goldens and tests unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
The reverify gate's compiled-output check refuses a fix that would change what a template compiles to when every planned edit is meant to leave the output alone. No test had seen it refuse anything: with the check disabled the whole mds-cli suite stayed green. No known `mds lint --fix` input reaches the refusal end to end, so these in-module tests drive it directly: - reverify_gate_refuses_a_candidate_that_changes_compiled_output: a gate built from a real, all output-neutral plan refuses a crafted candidate that still lints but compiles to different output, as an mds::io error with today's message. The plan's own fix, which keeps the output, is accepted. - reverify_gate_skips_the_output_check_for_a_plan_that_changes_output: a plan that also carries a legacy-interpolation edit accepts the same kind of candidate; the plan's output-neutral edits alone refuse it. - fix_pipeline_rejects_a_fix_that_would_change_compiled_output: a crafted output-neutral removal of a line of text comes out of run_fix_pipeline as Rejected, with the gate's message inside the reason `fix rejected:` prints. The real findings of the same source are fixed. Disabling the check fails all three; counting one output-neutral edit as enough fails the second; inverting the comparison fails the controls. Tests only: no product code, output or golden changes. Co-Authored-By: Claude <noreply@anthropic.com>
…not duplicate lines `no_raw_print_macro_outside_the_writer` requires a minimum number of `ewriteln!` / `ewrite!` calls across mds-cli/src. That number is a non-vacuity check only: it shows the scan found the registered writer macros at all, so the raw-print ban cannot pass by reading nothing. It is not a coverage count. Which files print through the writer is guarded file by file by `SITE_FLOORS`, with `lint.rs` pending in `FLOORS_PENDING` until its status lines move to the result sinks. At 80 the floor sat 5 under the crate's real count (85), so it had become a coverage count by accident. The lint refactor that follows gives stdin, a file argument and each directory entry one spine, which removes 16 duplicated status lines from lint.rs, and that alone would have failed the floor. Lower it to 60 and say in the comment what it guards. Planted control: pointing the scan at a macro name that does not exist fails the test with "non-vacuity: expected at least 60 `ewriteln!` / `ewrite!` calls across mds-cli/src, found 0". Co-Authored-By: Claude <noreply@anthropic.com>
…ntry Every input of `mds lint` now goes through one per-input spine, `lint_input`. It names the findings for the input, announces a capped result, then reports the findings, or runs the fix pipeline and applies `--fix` (`apply_fix`, with stdin's filter in `fix_stdin`) or previews it (`preview_fix`). It returns an `InputVerdict` (tally, would-fix, truncated): stdin and a file argument exit with its code, a directory counts it in its summary. The spine replaces the twin directory functions `lint_one_file_accumulating` and `lint_one_file_human`, and the report, fix and preview bodies of `run_lint_stdin` and `run_lint_file`, which now only load their input; a directory entry loads through `load_dir_entry`. `exit_by_severity` and `preview_exit_code` fold into `InputVerdict::exit_code`, and `emit_result` also collects a directory entry's findings into the directory's JSON document. Divergences kept, each an explicit branch: - A directory under human output reads each source first, so an oversized entry counts under "with errors" (exit 2) rather than resource-limited; JSON output reads a source only to fix it, after linting. - A directory's JSON document records a rewritten file's findings only once the rewrite succeeded; a file argument and a directory's human output show the residual first, then write. - A failed rewrite ends a file argument's run with the error (exit 2), prints `error writing` for a directory's human output, and becomes an error entry in a directory's JSON document. - `mds lint --fix -` stays a filter: its findings render against the source it emits, and it prints no `Fixed:`. - `Clean:` is printed for a file argument only, and a directory's status lines name the file (`<path>: fix rejected:`, the cap notice). No behaviour change: goldens and tests unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
lint_input's SourceText::Unread read-failure arm bound an intermediate `tally` variable only to move it, unchanged, into the following InputVerdict literal one line later. Inline the dir_entry_failed(...) call directly into the tally field, matching the sibling arm 12 lines above (which already inlines tally_from_result(&result)) and load_dir_entry's two analogous Err arms. No move-ordering hazard here (unlike load_dir_entry's other call site, where failure_tally(&e) must be evaluated before e is moved into dir_entry_failed, so that intermediate stays). Output, exit codes and control flow unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
…njudged A hold that outlasts the one-second deadline (a descheduled test thread) leaves only the deadline claim to check. That was said with eprintln!, which libtest discards on a pass, so the watch soak counted every run of this binary as unflagged and could not tell whether one iteration had judged the stricter claims. strict_claims_apply now takes the test's name and flags every run through common::record_run_flag: conclusive with "hold <d> vs deadline 1s; strict claims judged", inconclusive with "only the deadline claim judged". All 16 tests with stricter claims pass their own name (DependencyCase gains a `test` field, held_truncate_of_the_entry a parameter). The batch-deferral test, whose claims also need the other source's edit inside the deadline, flags through flag_strict_claims with that edit's offset in the line. RED: at the base, with MDS_TEST_FLAG_LOG set, the binary ran 23 passed and never created the flag log. After: 16 conclusive lines, 0 inconclusive. Mutation: TRUNCATE_HOLD planted at 1100 ms gave 16 inconclusive lines, 0 conclusive, all 23 still passing; restored with cp + cmp. Co-Authored-By: Claude <noreply@anthropic.com>
…ause preceded it Claim 1 (a rebuild is published while the writes go on) cannot be failed by a slow runner, but it can be satisfied without the cap: a writer that pauses for half a window lets a window close quietly, the watcher seeing each write up to half a window late. The module doc said no timing entered the claim, and an inconclusive run's flag spoke of the bound alone. The verdict is now judged on the cadence up to the read: the first rebuilt output was read before write k, and the claim counts as proven only when writes 1..=k (the last gap spans the read) hold no gap of half a window or more. Otherwise the flag says "claim 1 not judged" with that cadence. The new WriteCadence::first(n) gives the first writes as a record of their own. A conclusive run has no pause anywhere, so it has always proven the claim; that invariant is asserted. Two unit tests pin first() and the verdict's boundary: a read just after the pause is unjudged, one just before is proven. RED: with a planted 300 ms writer pause after write 3, the base run passed with "Hot v3!" read before write 4 and a flag that said only "the bound of 6 not judged". After: the flag reads "claim 1 not judged: "Hot v3!" read before write 4 of 719, after 1 gap(s) of 100ms or more in which a window could close quietly (4 writes over 335ms ...)". Unplanted: "claim 1 proven: "Hot v266!" read before write 267 ..., no gap of 100ms or more by then (267 writes over 2.017s ...)". Mutations, each alone, restored with cp + cmp: the verdict ignoring pauses fails claim_1_is_proven_only_when_no_pause_came_before_the_read at the read before write 11; first() ignoring n fails both unit tests. Co-Authored-By: Claude <noreply@anthropic.com>
Four watch tests took a count baseline after a fixed settle sleep and asserted equality later. A report from an earlier event that lands after the sleep (a slow runner, several events per save at --debounce 0) failed them exactly as the regressions they guard would. - watch_import_removal_stops_tracking_dep runs at --debounce 100, so the save that drops the import is one compile: wait_for_tap_count(warning, 1) is the baseline and the final count must be exactly that one. - watch_dir_mode_persistent_error_bounded_count runs at --debounce 100 and reads its baseline behind an ordered marker: a new source zz.mds that compiles. Its batch re-compiles every errored source, bad.mds first by name, so its "zz.md (0 deps)" line follows everything earlier events reported; compiling, it is a known source no tick rebuilds again. - watch_file_mode_entry_deleted_settles_then_recovers and watch_file_mode_parent_dir_deleted_bounded_errors_then_recovers run at --debounce 100 with one negative window of six ticks. Every report in file mode reports again, so no marker is exact there; instead the recovery's Recompiled line is the ordered anchor, and the not-found reports before it must number 1 to 2 (DELETION_REPORTS): the deletion's events gathered into one rebuild, plus at most one tick (the first tick's recovery, or one that saw the deletion before its events). The 200 ms settle before the kill and the total-count sanity checks go. RED (product planted with a 600 ms sleep at the top of rebuild_file and rebuild_dir_batch; the import test's save doubled and the dir test's a.mds touched at once, as an extra event): at the base the dir test failed "w1=1, w2=2" (cli_watch.rs:3473), the deleted-entry test "w1=1, w2=2" (:2981), the import test "left: 2, right: 1" (:1315); the parent-dir test, whose first window is 900 ms, failed under a 1000 ms sleep with "w1=0, w2=1" (:3811). The new tests passed under the same plants. Mutations, each alone in watch.rs, restored with cp + cmp: file mode keeping removed imports watched (foi extended, not replaced) fails the import test "left: 2, right: 1"; liveness_probe_file firing every tick while the entry is missing fails the deleted-entry test (7 reports) and the parent-dir test (6); the directory tick re-running errored sources fails the dir test "at the marker 3, after the window 7". Co-Authored-By: Claude <noreply@anthropic.com>
…action Three tests send SIGINT before `mds watch` installs its Ctrl+C handler and assert the default action, death by the signal: watch_dir_mode_ctrl_c_during_startup_compile_terminates, watch_file_mode_ctrl_c_during_startup_compile_terminates and the control arm of watch_readiness_handshake_makes_ctrl_c_exit_deterministic. A child inherits an ignored signal across exec and std::process::Command resets only SIGPIPE, so a test run started in the background of a non-interactive shell, which has SIGINT ignored, handed its watchers SIGINT ignored: the signal was discarded and the tests waited for an exit that never came. That is the earlier local failure of these three and no others. common::default_sigint(&mut Command) resets SIGINT to its default in the child (a pre_exec calling signal(SIGINT, SIG_DFL), with the same SAFETY note as limit_file_growth), and the three spawns go through it. Unix-only: a signal disposition is a unix notion, and Windows has no SIGINT a test can send one child. A new unix test, default_sigint_gives_a_child_sigint_s_default_action_though_it_would_inherit_it_ignored, plants the inherited disposition with a pre_exec of its own (no shell), then interrupts `mds fmt -` on a held-open stdin: with SIGINT ignored the child survives and exits once stdin closes (control); with default_sigint as well it dies of SIGINT. RED: a planted pre_exec ignoring SIGINT in the three spawns at the base failed all three after 20 s: "process must exit after SIGINT during startup compile" (cli_watch.rs:4795, :4895) and "control arm (SIGINT before the handler is installed): the process did not exit within 20s of the signal" (:5008). With default_sigint registered after the same plant, all three pass. Mutation, restored with cp + cmp: default_sigint as a no-op fails the new test, "got ExitStatus(unix_wait_status(0))", left None, right Some(2). Co-Authored-By: Claude <noreply@anthropic.com>
…ed bound The burst and cap tests judged their rebuild bound only when the writer never paused for half a window; a run with such a pause logged the bound and passed. common::most_debounce_windows already counts a window for every measured pause of half a window, so the bound holds in a slow run too, and asserting it there still catches a debounce switched off (about a rebuild per write) on exactly the runs that today judge nothing. - watch_debounce_single_rebuild_from_burst: the conclusive branch keeps "exactly one"; an inconclusive run must stay within most_debounce_windows + ALLOWANCE (one window for a watcher descheduled for half a window while the writer was not, as the cap test allows). - watch_debounce_cap_rebuilds_while_writes_never_stop asserts rebuilds <= most in every run. Its claim-3 doc says so: every run judges the bound, a paused writer only loosens it, and that looser bound no longer tells a cap that ends windows early, hence the inconclusive flag. The flag reads "at most N allowed, loosened by K gap(s) ...". RED (product planted: clamp_debounce always None, i.e. debounce off; each run forced inconclusive by a planted writer pause, 200 ms after write 6 of the burst, 300 ms after write 3 of the cap stream): at the base both passed, flagged "12 rebuild(s), not judged: ... allow up to 2" and "600 rebuild(s), the bound of 6 not judged". After: the burst fails "allows at most 3 rebuilds ... got 12" (cli_watch.rs:1830) and the cap test "allow at most 6 rebuilds ... got 574" (cli_watch_cap.rs:301). Control, the product restored with the same forced pauses: both pass, inconclusive, "1 rebuild(s) ... allow at most 3" and "4 rebuild(s), at most 6 allowed". Every plant restored with cp + cmp. Co-Authored-By: Claude <noreply@anthropic.com>
…ver merge their lines
common::append_line wrote a flag with writeln!, which is two writes: the
line, then its newline. The tests of one binary run at once, each opening
the soak's flag log for itself, and since the truncate binary now flags 16
tests per run, one writer's line fell between another's line and newline:
a stress run of the cap and truncate binaries (5 iterations, 85 flags)
logged 82 lines, three of them two flags merged ("...strict claims
judgedan_imported_partial_...: conclusive: ...") and three empty. The
soak's per-iteration verdict survived (a merged line still matches), but
its counts did not.
The line and its newline now go out in one write_all, which an appending
file lands whole at its end. A new test, flags_appended_at_once_stay_one_
per_line, has eight threads append 200 flags each to one log and requires
1600 well-formed lines.
RED: against the old append_line it failed "1600 lines for 1600 flags;
first malformed: ["writer_4: conclusive: flag 0writer_3: conclusive: flag
0", "", ...]". After: 20/20 stress iterations pass, and the same cap +
truncate stress run logs 85 well-formed lines, 85 conclusive, none
malformed.
Co-Authored-By: Claude <noreply@anthropic.com>
Eleven tests that were unix-only only because they made their links with the unix call now make them through common::make_symlink / common::remove_symlink and run on every OS, with every typed argument and every expected path in the platform's separator: - anchored_writes.rs: a link planted below --out-dir, a symlinked anchor the user named (its arguments now typed natively), build.output_dir committed as a link, and watch's rebuild, removal and out-dir checks through a link. - cli_watch.rs: an entry through a retargeted directory link, a directory through a retargeted link, and a source under a subdirectory swapped for a link. The directory test types `link/a` on every OS and keeps `link/..` as a unix-only row: Windows takes `link\..` back to the directory the link is in before it looks at the link, and the link itself typed as the directory is refused by design. - path_labels.rs: an out-dir and a watched path reached through a link are named by the link. The working-directory test makes its link through make_symlink too but stays unix-only: Windows cannot delete a process's working directory. Test-only; no product code changed. No RED applies (no behaviour change). Test-side controls, each planted in a copy and restored with cp + cmp: a real directory where each link was makes every ported test fail at its refusal or through-the-link check (exit code, refusal text, landed path, missing wait line); a retarget back to the watched directory fails the three watch refusal tests; a file written through the working-directory link fails its absence check; typing the link's target fails the banner. A type error planted in one ported test per file fails the Windows cross-clippy with E0308 in anchored_writes, cli_watch and path_labels. Co-Authored-By: Claude <noreply@anthropic.com>
Every lint golden records how many temporary-directory spellings its cell's streams had replaced by `$TMP`, and how many atomic-write temp names were replaced by `.mds-tmp-RANDOM.tmp`. All of them are zero, but nothing said they must be: a path leak that came back and was regenerated with MDS_GOLDEN_PRINT=1 would have been recorded as the new truth, and every test would still pass. `no_golden_records_a_leaked_path` now holds both counts at zero in every golden of both tables, and fails if any stored string holds either replacement token, so a returned leak needs an edit to the test, not just to the data. Its controls normalize a stream naming a real fixture tempdir, and one naming an atomic-write temp file, exactly as a cell's streams are, and require each to be reported; the same stream naming neither is not. The harness doc says so; `TMP_NAME_REPLACEMENT` is now public for the check. Before (the gap): golden 1 of single.rs planted with `tmp_paths: 1` passed golden_storage_stays_within_bounds, every_cell_has_exactly_one_golden_and_no_golden_is_dead and matrix_has_the_literal_cell_counts (3 passed). Mutation controls, each planted alone in the data and restored: - single.rs golden 1 `tmp_paths: 1` fails it: "single.rs golden 1: 1 temporary-directory path(s) replaced by $TMP" - `$TMP/d: ` prefixed to dir.rs golden 0's stderr fails it: "dir.rs golden 0: a stored string holds $TMP: ..." - dir.rs golden 1 `tmp_names: 1` fails it: "dir.rs golden 1: 1 atomic-write temp name(s) replaced by .mds-tmp-RANDOM.tmp" Co-Authored-By: Claude <noreply@anthropic.com>
On Windows the golden normalizer rewrites every backslash in a cell's
output to `/`, so no lint golden can see which separator `mds` printed
there: a separator it got wrong would pass every cell. The harness doc
said what the rule does, not what it waives.
The harness doc now names the waiver and the tests that pin the
separators instead, reading output as `mds` printed it: in cli_lint.rs a
directory entry's `Fixed:` line and its cap notice in the platform's
separator; in lint.rs `relative_display`'s unit tests for the JSON
`files[].file` key, `/`-separated on every OS; and
`dir_fixture_markers_hold`.
That marker test parsed its JSON report from the normalized stdout, so
on Windows it could not see a `\` key either. `run_dir` now returns a
`DirRun`: the normalized record (unchanged, still what the goldens and
the double-run check compare) and the stdout exactly as printed, which
`dir_fixture_markers_hold` parses. `normalizer_rewrites_every_tempdir_spelling`
pins the waiver itself: on Windows a printed `\` (escaped in JSON or
not) reads as `/` once normalized; elsewhere it is kept.
Controls (test-side, in the harness, each restored with cp + cmp):
- A JSON key printed as `api\\x.mds` (planted into the run's stdout
before normalization) with the Windows rewrite forced on every OS:
before this change dir_fixture_markers_hold PASSED (the gap); after
it, it fails: `left: [..., "api\\x.mds", ...]` vs
`right: [..., "api/x.mds", ...]`.
- The same key planted without the forced rewrite failed before the
change too, so the plant is live.
- The rewrite forced on alone makes the normalizer test's new
assertion fail on macOS (`left: "{\"file\":\"api/x.mds\"} d/x.mds"`),
which is exactly the value its Windows branch expects.
Co-Authored-By: Claude <noreply@anthropic.com>
The guard's module doc said the path-sink rule finds "at least 150 message sinks and 25 causes", while the test asserts at least 19 causes (lowered when every write and removal failure came to be worded once). Both floors are now named constants, `MESSAGE_SINK_FLOOR` (150) and `CAUSE_FLOOR` (19, with the reason the inline comment gave), which the assertions and their messages use and the module doc links to, so the doc can no longer state a floor the test does not hold. Controls, each planted alone and restored with cp + cmp: - CAUSE_FLOOR 1000 fails messages_interpolate_an_error_only_as_its_cause: "expected at least 1000 causes shown through [\"io_cause\", \"notify_cause\"], found 21". - MESSAGE_SINK_FLOOR 100000 fails it: "expected at least 100000 message sinks across mds-cli/src, found 203". Co-Authored-By: Claude <noreply@anthropic.com>
…ites `write_atomic`'s doc called its Windows sharing-violation caveat developer-machine only, because "CI runs this suite on ubuntu". The Windows Rust job runs cli_watch, cli_watch_cap (hundreds of rename-overs in a run) and cli_watch_truncate through this helper, so a `cannot rename` panic there would have been read as impossible. The doc now says what holds on Windows: a rename over a file another process holds open succeeds when that process opened it as std does, sharing deletion (FILE_SHARE_DELETE), since std's `rename` falls back to a POSIX-semantics replace; a handle opened without that flag (a virus scanner's, say) makes it fail with a sharing violation; and the Windows CI job runs those suites, so such a panic there is a failure to triage. Doc comment only; no behaviour change. Co-Authored-By: Claude <noreply@anthropic.com>
`each_catch_wraps_one_compile_call_and_nothing_else` built its planted
sources with three closures (`replaced`, `appended`, `defined`), each
repeating the same block: copy the sources, find the planted file or
fail on the precondition, change its text, return the copy.
`the_hook_writes_one_constant_and_nothing_about_the_panic` held a fourth
copy of it in its own `plant` closure.
That block is now one helper, `plant_in(sources, file, edit)`, and each
closure says only what it changes. Pure refactor of the test: the same
plants, preconditions and messages; panic_hook still runs 25 tests and
every one passes.
Control (test-side, restored with cp + cmp): `plant_in` made to drop
its edit fails both tests — each_catch_wraps_one_compile_call_and_nothing_else
misses all 15 of its plants ("each of these must be reported"), and
the_hook_writes_one_constant_and_nothing_about_the_panic misses its
machinery plants ("missed: [\"a `resume_unwind` in main.rs\", ...]") —
so every plant still goes through the helper.
Co-Authored-By: Claude <noreply@anthropic.com>
The write-funnel guard listed symlink creation among what it does not
see: the bare `symlink(` is part of `is_symlink(` and `check_symlink(`,
which production code calls to read a file's type. But no spelling of
it was needled, so a raw symlink made anywhere in mds-cli's sources
passed.
NEEDLES gains `fs::symlink(` (std's `std::os::unix::fs::symlink(` and
rustix's `fs::symlink(` alike), `symlinkat(` (rustix, relative to a
descriptor), `soft_link(` (std's deprecated by-path form),
`symlink_file(` and `symlink_dir(` (Windows); none is part of another,
so each call is counted once. Product code makes no symlink, so no
licence is added and the real-tree scan stays clean. The "does NOT
see" list now names only the bare name imported with `use`, and says
every link the suite makes is made by test code (`#[cfg(test)]` items,
or `tests/` such as `common::make_symlink`, which the scan does not
read); the module and NEEDLES docs list the new calls.
RED (plants added before the needles):
write_funnel.rs:473:9 must be flagged:
fn f(a: &Path, b: &Path) { let _ = std::os::unix::fs::symlink(a, b); }
left: 0 right: 1
Controls, all test-side, each planted alone in tests/write_funnel.rs and
restored with cp + cmp:
- Each needle dropped in turn fails its own plant with `left: 0`:
`fs::symlink(` (std::os::unix::fs::symlink), `symlinkat(`
(rustix::fs::symlinkat), `soft_link(` (std::fs::soft_link),
`symlink_file(` and `symlink_dir(` (std::os::windows::fs).
- The real-tree scan with test-only items left unblanked (and the
licence-drift check silenced so the violations print) reports the
links the crate's own tests make: build.rs 2, output.rs 1, write.rs 11
`fs::symlink(`, and write.rs's `symlink_dir(` and `symlink_file(` —
so the needles see the spellings the tree really uses.
- The needle made bare (`symlink(`) fails the new negative control
(`left: 3`: `is_symlink(`, `check_symlink(`, `make_symlink(`) and the
real-tree scan with 13 false hits, every one a production
`is_symlink(` or `check_symlink(` call (build.rs 1, lint.rs 1,
output.rs 2, watch.rs 3, write.rs 6), which is why the needle is
qualified.
Co-Authored-By: Claude <noreply@anthropic.com>
…ry tick Directory watch's idle tick watched the --vars file's directory outside the root again on every tick, and the root a second time on the first tick, though the startup had armed both. notify's watch() of a path it already watches is no no-op: on macOS it restarts the FSEvents stream, dropping the events that land meanwhile, and on Windows it opens another directory handle without stopping the old one's read, so each tick leaked a handle and every event in that directory was delivered once more for each tick elapsed. The --vars directory now joins the set of directories outside the root that the watcher holds (renamed from armed_external_dirs to armed_dirs, the name file mode gives the same set), seeded by the startup only once its watch is in place. The tick watches it only while the set does not hold it - its startup watch failed, or it vanished and came back - and drops it from the set, with no unwatch, while it is gone. A dependency directory that is also the --vars directory is one entry: the dependency's arm finds it held and does not watch it a second time, and the tick's unwatch of directories no source imports any more leaves it alone, since the vars file still needs it. The first tick is no longer a reason to re-arm the root; its reconcile walk does not depend on it. liveness_probe_dir and arm_external_dirs_after_rebuild take the watcher as &mut dyn Watcher, so a unit test can count what a tick asks of it. RED (before the fix, field renamed only): - a_tick_watches_a_directory_only_when_the_watcher_does_not_hold_it: left: [Watch(<root>), Watch(<ext>), Watch(<vars dir>) x6] right: [Watch(<ext>), Watch(<vars dir>)] - a_vars_directory_a_dependency_shares_stays_watched_once_the_import_goes: left: [Watch(<shared>)] right: [] Mutations (each planted alone, observed failing, restored): - first_tick back in the root re-arm condition: the first test gets Watch(<root>) first. - the vars directory watched without the held check: both tests fail. - the vars directory not excepted from the unwatch of unimported directories: both tests fail (Unwatch then Watch of it). - a vanished vars directory left in the set: the first test's reappearance step gets no Watch. Co-Authored-By: Claude <noreply@anthropic.com>
…se event was lost
Directory watch's idle tick diffed the sources and their dependencies
against the baseline, but not the --vars file, though the baseline holds
its stamp: a --vars edit whose event was lost - to an inotify queue
overflow, or in the gap of an FSEvents stream restart - left every output
compiled with the old values until something else rebuilt it, and the next
batch's re-baseline took the new stamp in unseen. File mode's tick already
looks at its --vars file. The tick now diffs the watched set (sources,
dependencies and the --vars file); a change found to the vars file is
taken out of the batch and rebuilt as a vars change, which recompiles every
source, as the batch of its event does.
The event handler tested whether the vars file changed only after it had
dropped the paths below the out-dir and in hidden directories and
node_modules/ below the root, so a --vars file there (root/.config/v.json)
had every event dropped and an edit to it rebuilt nothing. It now tests
first.
tracked_set is now used by tests alone and is compiled for them only; its
description moves to tracked_paths, which the product uses. The doc
comments that said the tick leaves the vars file to its event now say it
diffs the watched set. The arming test that let a vars change rebuild a
directory with an imaginary dependency is split in two: a vars rebuild
recomputes the dependency directories from what the sources import, so
the vanish-and-return half now runs without one.
RED (at the previous commit):
- an_idle_tick_rebuilds_with_a_vars_edit_whose_event_was_lost:
"the tick rebuilds with the vars file as it now reads; out/a.md:
Some("A one\n")"
- a_vars_file_s_event_rebuilds_where_other_paths_events_are_dropped:
".config: the vars file's event rebuilds"
Mutations (each planted alone, observed failing, restored):
- the tick diffs the tracked paths only: the tick test fails (A one).
- the tick's vars change not passed to the rebuild: the tick test fails.
- the vars check made after the hidden/node_modules filter: the event
test fails at ".config".
- the vars check made after the out-dir filter: the event test fails at
"the out-dir".
Co-Authored-By: Claude <noreply@anthropic.com>
…mported is unwatched A rebuild arms the directory of a dependency outside the root that its compile reports first, but only the idle tick unwatched one that no source imports any more, and --poll-interval 0 runs no tick: every such directory the session ever imported from stayed watched for as long as it ran - on Linux, inotify watches counted against fs.inotify.max_user_watches, which that unwatch exists to protect. The tick's unwatch moves into one helper, unwatch_unimported_dirs, which both the tick and arm_external_dirs_after_rebuild - the step every rebuild ends with - call: it unwatches each directory the watcher holds that is no longer imported, leaving the --vars file's directory, which the watcher holds for that file whatever the sources import. On a tick it now finds nothing to do unless the two disagree. The comment in process_dir_batch that said the tick drops the stale directories now says the rebuild does. RED (at the previous commit): - a_rebuild_unwatches_a_directory_no_source_imports_any_more: left: [] right: [Unwatch(<gone>)] Mutations (each planted alone, observed failing, restored): - the rebuild's unwatch call removed: the RED above. - the --vars directory not excepted: this test and the shared-directory test both fail (the vars directory unwatched too). - the helper blind to what is imported: this test fails (the still imported directory unwatched too). Co-Authored-By: Claude <noreply@anthropic.com>
…a failing one warns once
After every rebuild, arm_external_dirs_after_rebuild statted each
dependency directory outside the root before it checked whether the
watcher already held it, so every armed directory cost a stat per batch.
And a directory that exists but cannot be watched - EACCES, inotify's
max_user_watches spent - was tried again after every rebuild and printed
"warning: failed to watch external dep dir" after every one, for the rest
of the session.
The rebuild now passes over a held directory before it looks at it. The
arming of one directory moves into arm_dir, which returns a DirArm: Held,
Armed, Failed (the first failure of a run, for the caller to report) or
FailedAgain (no watch of it succeeded since: reported already) - the shape
StdoutOutcome gives stdout's failures. The run is kept in a new
LivenessState field, failing_dirs, seeded by the startup's arm; it ends when
a watch of the directory succeeds - a rebuild's or the idle tick's - or when
no source imports from the directory any more, so an import that brings it
back reports its failure again.
RED:
- a_directory_whose_watch_keeps_failing_is_reported_once_until_a_watch_succeeds,
against arm_dir with the old rule (every failure reported):
"the first failure is reported, the two after it are not:
[Failed(..refused..), Failed(..refused..), Failed(..refused..)]"
- a_directory_imported_again_reports_its_failed_watch_again, before the
unimported directories' runs were dropped: "the directory imported again
reports its failed watch again: FailedAgain"
Mutations (each planted alone, observed failing, restored):
- a successful watch in arm_dir leaves the run open: the first test fails
("a failure after a watch that succeeded is reported again: FailedAgain").
- every failure reported: the first two tests fail.
- the unimported directories' runs kept: the second test fails.
- the tick's successful watch leaves the run open:
a_tick_that_arms_a_failing_directory_ends_its_run_of_failures fails
("a failure after the tick's watch succeeded is reported: FailedAgain").
The held-before-stat order has no test: it changes no outcome, only the
cost.
Co-Authored-By: Claude <noreply@anthropic.com>
…ed or was held A directory watch's rebuild for a --vars edit recompiles every source with the dependency graph cleared, so that the records of paths that are no sources any more go. A source whose compile failed went through record_error, which keeps the last dependency set, but after the clear it kept an empty one; a source held because a file its compile read was emptied mid-save returned with no entry at all. A dependency outside the directory argument that only such a source imported left the tracked set and external_dep_dirs (which only a successful compile refilled), so the rebuild's arming unwatched its directory at once - under --poll-interval 0 too - and its events were dropped: the edit that fixed the source was not seen until an unrelated change rebuilt it. An error path must not narrow what is watched; an incremental batch already kept the dependencies. The vars pass now takes the previous graph and seeds each source's entry from it just before the source is compiled: a successful compile records its fresh dependencies in their place, a failed or held one keeps them, and a source retired as deleted drops its seeded entry. The late look of a failed compile, which takes the files it read to be the source's last dependencies, therefore covers them again, so a compile that failed on a dependency emptied mid-save is held instead of reported. external_dep_dirs is recomputed from the graph at the end of the pass, through a new DirWatchState::dep_dirs_outside that the incremental pass's prune now uses too. RED (at abbcf7d, the test alone): a failed compile: the dependency outside the root is still tracked; tracked: {".../root/page.mds"} a compile held on its source: the dependency outside the root is still tracked; tracked: {".../root/page.mds"} a compile failed on its dependency: the source is held, not reported; held: HeldBatch { paths: {}, vars_changed: false }, errored: {".../root/page.mds"} Mutations (each planted alone, observed failing, restored with cp + cmp): - seeding disabled: the new test fails (a failed compile: ... still tracked). - the vars pass's recompute dropped: a_batch_forgets_a_directory_no_ source_imports_any_more fails (vars changed: true: the directory no source imports from is forgotten). The first form of the test missed it, so the guard test was added before commit. - the incremental pass's recompute dropped: the same test fails (vars changed: false); the unit suite alone did not catch it before. Tests: a_vars_change_keeps_the_dependencies_of_a_source_that_failed_or_ was_held (three cases, through process_dir_batch past the look, plus the rebuild's arming with a recording watcher) and a_batch_forgets_a_ directory_no_source_imports_any_more (both kinds of batch), on a shared cross_root_page fixture. Co-Authored-By: Claude <noreply@anthropic.com>
…with a notice A directory watch retires a deleted source's outputs only through the record of where the session wrote them. A source with no record - its compile, or every write, failed for the whole session - went straight to being forgotten and printed nothing, so a draft that never compiled, beside an out/draft.md an earlier mds build left, was deleted and its output kept with no line at all, against the rule that anything kept is kept with one notice saying why. retire_deleted now takes the watched root and the output base. A source with no record, still known to the session (walked, or errored), below the root and no partial, has its outputs where a write would put them (output_path_for, which never takes its flattened arm there); each is retired through the same rule, and since the session wrote nothing there for it, nothing is removed and each file present is kept with "Kept <output>: not written by this session" (or "... by this source" when the session wrote it for another source), quiet-aware. A source already forgotten by an earlier batch of the same deletion tells nothing again. spec.md says so. RED (at 9f59a0e, the test alone), both kinds of batch: a deletion alone: one notice for each file kept ... left: [] right: ["Kept notes/draft.json: not written by this session", "Kept notes/draft.md: not written by this session"] a deletion with a --vars edit: the same. Mutations (each planted alone, observed failing, restored with cp + cmp): - the still-known guard bypassed: the new test and watch_keeps_the_hand_written_siblings_of_a_deleted_source fail with the notice repeated for every later event of the deletion (the first form of the fix had this defect; the guard was added before commit). - the partial exclusion dropped: the siblings test fails with "Kept notes/_p.md: not written by this session". Test: watch_keeps_with_a_notice_the_outputs_of_a_deleted_source_it_never_ wrote (positive control: ok.md, which the session wrote, is removed). Co-Authored-By: Claude <noreply@anthropic.com>
…cannot be looked at
retire_output took every failed look at the output - a directory on its
way that may not be searched (EACCES), a link loop, an I/O error - for
"the file is gone", and its callers then dropped the session's record of
the file: the directory watch's retire and file mode's change-of-kind
retirement both remove the last_written entry on a true return. The file
may still be there, and the session stopped treating it as its own with
nothing reported; a later change of kind would then call it "not written
by this session". An error path must not change what the session tracks.
Only a name that is not there - NotFound, or NotADirectory for a file
where a directory on its way should be - is gone now. Any other failed
look goes on as a file present: with the session's record, its removal is
tried through remove_proven, which reports the failure ("warning: could
not remove ...") and keeps the record; with none, the file is kept with
its notice as before.
RED (at 36fe411, the tests alone):
a_retired_output_that_cannot_be_looked_at_keeps_its_record:
an output that cannot be looked at keeps the session's record of it
(a_retired_output_that_is_not_there_is_gone_with_its_record passed at
base: it is the control for the two names that are gone.)
Mutations (each planted alone, observed failing, restored with cp + cmp;
this is the record-keeping look, not a removal proof: no mutation touched
remove_proven or its refusals):
- NotADirectory dropped from the gone kinds: "a file on its way: with no
record, the output is gone - not kept, with a notice" fails.
- NotFound dropped: "a missing name: ..." fails.
Tests: a_retired_output_that_is_not_there_is_gone_with_its_record (all
platforms) and a_retired_output_that_cannot_be_looked_at_keeps_its_record
(unix only: the look is refused through a directory's mode bits; skipped
with a reason when running as root).
Co-Authored-By: Claude <noreply@anthropic.com>
…hat shows something File mode re-reported the --vars file's duplicate-key warnings on every rebuild whose content_changed was true, decided before the write. After a change of kind, content_changed stays true, and a rebuild whose output is kept beside a file the session did not write prints its "Kept" notice only the first time: every later rebuild of the same - another event of the same save, a dependency edit with the same output - printed the warning and nothing else, breaking the once-per-observable-rebuild rule. On macOS one save of the kept source printed it four more times. The warning is now decided after the write, by what the rebuild shows: a written output (before its "Recompiled" line, as before), a "Kept" notice (after it, now; none when the rebuild before kept the same and nothing is told), or a failed write's report (before the report, as before). A repeat of a stdout failure, which reports nothing, and a closed stdout, which ends the session, print none. Directory mode already gated it on what its batch wrote. RED (at 76c5c8d, the test alone): the warning is printed at startup and with the notice - not again by the rebuild that tells nothing ... left: 6, right: 2 Mutations (each planted alone, observed failing, restored with cp + cmp): - the kept arm warns when told too: the new test fails (6 against 2). - the kept arm never warns: the new test fails (wait_for_tap_count: expected at least 2 occurrences). - the written arm does not warn: i16_file_watch_vars_file_duplicate_ warns_at_startup_and_on_every_rebuild fails (1 against 2). - the failed-write arm does not warn: watch_file_vars_duplicate_warning_ comes_with_a_reported_write_failure fails (1 against 5). Tests: watch_file_vars_duplicate_warning_skips_a_kept_rebuild_that_tells_ nothing (debug builds: the pause after a file rebuild's reads makes the rebuild of the same certain, with the source read as saved) and watch_file_vars_duplicate_warning_comes_with_a_reported_write_failure (a guard for the failed-write arm: one warning per report). The I16 test's doc line on where a rebuild prints the warning is reworded. Co-Authored-By: Claude <noreply@anthropic.com>
With --clear, the event handler of either mode cleared the terminal before it handed over to the rebuild, which only then decided to hold: a rebuild held while a watched file is empty prints nothing, yet the screen was wiped, for up to a second, and each further event during the hold wiped it again; the rebuild that ended the hold - its deadline, the idle tick, or the next event - never cleared, so its result landed on a screen cleared earlier. The README, spec and changelog say a held rebuild prints nothing. The clear is now asked for and made in two steps: an event's rebuild asks for it (PendingClear::ask, on the session state; file mode's event handler takes the pending clear as a parameter), and the rebuild makes it once no hold holds it back any more, just before anything it can print - after the look's verdict and after the late look of a file its compile or --vars load read emptied (file mode: before a --vars or compile failure's report, and before the route and write; directory mode: before a --vars failure's report, and before the batch's compiles). A held rebuild leaves the clear asked for, so the rebuild that ends the hold makes it, whatever runs that rebuild. The idle tick asks for none, as it never cleared. In directory mode a source held alone by its own compile does not hold the batch, whose other sources print. RED (against a no-change scaffold at 155ac9f: the clear asked for by the event handlers and made first thing in the rebuild, before the look, as the base made it): a_held_file_rebuild_leaves_the_clear_to_the_rebuild_that_ends_the_hold: its look: a held rebuild leaves the clear asked for a_held_directory_batch_leaves_the_clear_to_the_batch_that_ends_the_hold: --clear: true: the held batch leaves the clear its event asked for Mutations (each planted alone, observed failing, restored with cp + cmp): - file mode's clear made before the look: its look case fails. - file mode's clear made right after the look's verdict, before the late look: its compile case fails (a held rebuild leaves the clear asked for). - directory mode's clear made before the look: --clear: true case fails. - each of the five places the clear is made, removed in turn: the file --vars and compile failure cases, the file hold-ending case, the directory --vars failure case and the directory hold-ending case fail. - either event handler no longer asking: the file event test and the directory held-batch test fail. Tests (pure, no terminal: the clear's own TTY check is unchanged): a_clear_asked_for_is_made_once, a_file_event_s_rebuild_asks_for_the_ clear_only_under_clear, a_held_file_rebuild_leaves_the_clear_to_the_ rebuild_that_ends_the_hold, a_rebuild_that_reports_a_failure_makes_the_ clear, a_held_directory_batch_leaves_the_clear_to_the_batch_that_ends_ the_hold. Co-Authored-By: Claude <noreply@anthropic.com>
…whatever a rebuild shows A directory batch printed the --vars file's duplicate-key warning only when one of its sources wrote an output, and only after the whole batch, after every Recompiled line. File mode prints it once for each rebuild that shows something: with a written output, before its Recompiled line; with a Kept notice a change of kind prints, after the notice; and with a failed write's report, before the report - never for a rebuild that shows nothing. A directory batch whose source was kept with a notice, or whose write failed and was reported, printed no warning at all. The batch now hands its sources a BatchVars - the variables every compile takes, and the resolved vars whose duplicate keys are still to be reported - and the first source that shows something reports them: before its Recompiled line, after a Kept notice it prints (a kept rebuild that keeps the same again and tells nothing reports none), or before its failed write's report. A compile failure, an unchanged output, a held source and a partial show nothing for it, as in file mode. The warning prints once per batch however many sources show something, and --quiet prints none, as the emitter already did. process_dir_batch and its two passes no longer return whether anything changed: nothing reads it now. RED (6ba13aa, new tests alone): watch_dir_vars_duplicate_warning_comes_once_before_a_batch_s_recompiled_lines: the batch's warning comes before its first `Recompiled` line left: 1, right: 2 watch_dir_vars_duplicate_warning_skips_a_kept_rebuild_that_tells_nothing: wait_for_tap_count ... expected at least 2 occurrences of "warning: key 'x' is set more than once in vars file ..." within 2s; saw 1 watch_dir_vars_duplicate_warning_comes_with_a_reported_write_failure: the warning is printed at startup and with each failed write's report left: 1, right: 6 Mutations (each planted alone, observed failing, restored with cp + cmp): - the written arm reports nothing: the batch test (1 vs 2), I17 and I19 fail. - the kept arm never reports: the kept test fails (saw 1). - the kept arm reports even when it told nothing: the kept test fails (7 vs 2). - the failed-write arm reports nothing: the write-failure test fails (1 vs 6). - the report made by every source, not once per batch: the batch test fails (3 vs 2). - the report made after the Recompiled line: the batch test's order assertion fails (1 vs 2). - a compile failure reports too: the batch, kept, I17 and I19 tests fail. Co-Authored-By: Claude <noreply@anthropic.com>
watch_dir_vars_duplicate_warning_comes_once_before_a_batch_s_recompiled_lines failed 2 of 10 iterations of `cargo nextest run -p mds-cli --test cli_watch --stress-count 10` at its own commit (and once in a full `nextest run -p mds-cli`), each time with the warning printed three times: once at startup, then once before `Recompiled notes/a.md` and again before `Recompiled notes/b.md`. That is two batches, each printing the warning once, as designed: a late event for a fixture write made before the spawn (FSEvents delivers those after the session goes live) rebuilt a.mds alone after the test had edited the vars file, so a.md changed in a batch of its own; the vars file's own batch then found only b.md changed. A test-side control reproduces it every time: saving a.mds unchanged just before the vars edit failed 10 of 10 iterations of the test with the same three warnings, in the same order. The test now settles queued events first (settle_queued_events, a barrier source whose failed compile it waits for), as the other tests that edit right after the startup do. With the same planted save before the barrier: 10 of 10 passed. The full cli_watch binary under --stress-count 20: 20 of 20 passed. No product change; the claims and their controls are unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
CompileFailure::unreported() returns exactly the error that still has to be reported (none for a panic, which the panic hook reported), but its name reads as "not reported" and misled a review note into saying such a failure is silent. Rename it into_report(), the by-value accessor's usual name, at its five product call sites and in two unit tests (four calls). No behaviour change; BatchVars' own unreported field (the duplicate keys no source of the batch has reported yet) keeps its name. Co-Authored-By: Claude <noreply@anthropic.com>
EmptyHold::on_rebuild and EmptyHold::on_late_empty change the hold and
return the HoldVerdict that decides whether the rebuild compiles; a
call that dropped the verdict compiled silently and would compile
against an emptied file. Inputs::and_found_from is a by-value builder:
a dropped result silently loses the search for the mds.json nearest a
source, which the refusal to write an output over a file the run reads
depends on. Every current call site uses both; this makes the next edit
that does not a compile error.
- HoldVerdict is #[must_use] (it covers both producers).
- Inputs::and_found_from is #[must_use = "the search is part of the
inputs only in the value returned"].
RED (test-side plants in the two modules' tests: a bare
`hold.on_rebuild(false, t0);`, `EmptyHold::Expired.on_late_empty(t0);`
and `Inputs::default().and_found_from(&path("page.mds"), beside);`):
before the attributes, `cargo +1.98 clippy -p mds-cli --all-targets --
-D warnings` exits 0 with all three in place. After them, the same
plants fail it, exit 101: "unused `watch::HoldVerdict` that must be
used" twice and "unused return value of `write::Inputs::and_found_from`
that must be used" with the note "the search is part of the inputs only
in the value returned". The plants are not committed.
Co-Authored-By: Claude <noreply@anthropic.com>
The watch module's key invariants still said "Exit 0 on clean Ctrl+C; non-zero only on startup failure". Since #389 a session that caught a compile panic keeps watching and exits 101 when it is stopped (the exit funnel's final_exit_code returns 101 first, under either policy), and a panic anywhere else ends the session with 101, as spec section 5 "Panics on the CLI" says. The bullet now names both; startup failure remains the only other non-zero exit. Comment only. Co-Authored-By: Claude <noreply@anthropic.com>
…eplaced The Windows arm of the write primitive moved its temporary file over the target with tempfile's `persist`, a `MoveFileExW(MOVEFILE_REPLACE_EXISTING)` alone. That fails with ERROR_ACCESS_DENIED (os error 5) while any other handle is open on the target, even one that shares it for delete: an editor, a viewer, a virus scanner, an indexer, or a test polling the output. Every output, `.map` sidecar, `fmt` / `lint --fix` rewrite and `init --force` starter written over a held file failed, and `mds watch` reported it and wrote nothing until the next change. This is the likely cause of the Windows CI failure of panic_hook's `a_watched_directory_goes_on_past_a_source_whose_compile_panics` (its 20 s wait polls the output every 5 ms while the watcher replaces it). The replace now renames with `std::fs::rename`. Its first step is the same `MoveFileExW`; on ERROR_ACCESS_DENIED it retries with `FileRenameInfoEx` and REPLACE_IF_EXISTS | POSIX_SEMANTICS, which replaces a target held open with FILE_SHARE_DELETE (std's default share mode). Always using std's rename was chosen over "persist, then fall back on os error 5": persist's failure path re-marks the file FILE_ATTRIBUTE_TEMPORARY, so a fallback would land a file still marked temporary, and the fallback is std's own second step anyway. For the same reason the temporary file is now created with `make_in` and a plain `OpenOptions::create_new` open instead of `tempfile_in`, which marks its file temporary and leaves only tempfile's own moves to clear the mark: a rename by std would leave an output, or a rewritten source, marked temporary for good. The temporary file is still deleted as it drops on any failure before the rename; after the rename its cleanup is disabled, so nothing at the old name is deleted. The new-file commit (`persist_noclobber`) is unchanged. No new dependency; Cargo.lock unchanged (178). Tests: - write.rs, Windows only: `on_windows_a_file_another_program_holds_open_is_still_replaced`. Control arm: tempfile's `persist` over a target held by `File::open` must fail with raw os error 5 (the test fails if it succeeds); product arms: `write_compiled` and then `replace_if_unchanged` (Fsync) over a held target each return Ok, a fresh read by path finds the new bytes, and no `.mds-tmp-*` is left. - cli_watch.rs, every OS: `watch_replaces_an_output_another_program_holds_open` holds the output open across a source edit and waits for the rebuild; its failure messages carry the watcher's stderr. Unix renames over an open file, so it passes there either way. - The existing every-OS `atomic_write_file_directory_target_refused_without_residue` now runs the new rename's failure path on Windows (a file over a directory fails; no temporary file left). RED: not run locally (macOS and Linux rename over an open file); RED in phase via Windows CI, where the control arm is the hazard's evidence. Windows cross-clippy compiles the new cfg(windows) test (planted `let _: u8 = "planted";` there: E0308 at write.rs:4646; restored with cp + cmp). write_funnel: needle `make_in(`; write.rs licences `tempfile_in(` and `.persist(` removed, `make_in(` 1, `.create_new(` 1 and `fs::rename(` 1 added; the primitive pin now requires exactly one `.sync_all()` and exactly one `fs::rename(` in `mod windows`, so with the exact licence count the one by-path rename in write.rs is the Windows arm's. Synthetic controls: a planted `make_in(` is flagged; the pin flags a Windows arm without the rename, with `persist` in its place, or with a second rename. Test-side controls on the real file: the three new licences set to 0 -> both scans fail naming `make_in(`, `.create_new(` and `fs::rename(` in write.rs; the pin's needle set back to `.persist(` -> `mod windows: 1 .sync_all() call(s) and 0 fs::rename( call(s)`; each restored with cp + cmp. The watch test's wait was shown to bite with a planted wrong expected text (restored). Co-Authored-By: Claude <noreply@anthropic.com>
…cher printed
Windows CI failed `default_output::a_watched_directory_goes_on_past_a_source_whose_compile_panics`
at its "control: the edit rebuilds" wait with nothing but the assertion's
own words: the watcher's stderr was never printed, so a write it reported
failing could not be told from an event it never saw.
The watch tests' `wait_for_file` (a bool every caller wrapped in `assert!`)
becomes `assert_file_comes_to_hold(path, text, &stderr, what)`, which fails
at the caller (`#[track_caller]`) with what the file holds by then and the
watcher's stderr so far. All eight waits use it: both tests of a watch
session past a compile that panics (file and directory, control and
triggered legs) and the file-event callback test's control leg, for each
of its fixtures. No test's behaviour or bound (20 s) changes.
Control: a wrong expected text planted once in the directory test's "an
edit after the panic rebuilds" wait fails with
an edit after the panic rebuilds: <tmp>/d/b.md did not come to hold
"Hello planted b\n" within 20s (it holds Ok("Hello again b\n")); the
watcher's stderr so far:
mds: internal compiler error
note: this is a bug in mds; please report it at ...
at panic_hook.rs:1134:9 (the caller); restored with cp + cmp.
Co-Authored-By: Claude <noreply@anthropic.com>
… not after reading it
A source joins the session's known files only once a compile of it
succeeds, so the look a directory batch takes before it reads anything
did not stat a source the batch named for the first time - one just
created - and the batch's re-baseline stamped it after its compile had
read it. A truncation landing between the look and the read was never
found: the compile read the source empty and the batch published the
empty output. A save in that window was absorbed by the stamp taken
after the read, where the next look should see it again.
The batch's look now also stamps every path the batch names and every
path an earlier batch held back, a source no compile has read yet
included, and the re-baseline keeps the look's stamps for the paths
held back as it keeps them for the watched files. A new source
truncated after the look is found emptied once its compile has read it
and is held under the existing hold, with the same fixed one-second
deadline; the next batch - the truncation's own, or one with nothing
new - finds it emptied again and holds at that same deadline, and once
the source is written one rebuild publishes it. A save after the look
leaves the source differing from the baseline, so the next look sees
it.
File mode needs no change: a file rebuild starts only on an event for
a file of interest and its look stats exactly those files, so every
path it knows of before its compile is stamped by the look; a newly
imported dependency is known only from the compile that read it.
The unit test whose source stood in for a file emptied between the
look and the read now uses an included module instead, since the look
stats the source itself.
Before the fix:
- cli_watch_truncate.rs:1129:5: directory mode, a source created and
truncated after its batch looked: an empty output was published
7.653167ms after the truncation, before the 1s deadline could have
passed; states read: ["", "N two\n"]
- watch.rs:10339:9: a held source emptied before the hold ends is
held, not published empty (left: Some(""), right: None)
The test of a save after the look passes before and after the fix:
the compile reads the save either way, so the outputs differ only when
the save's own event is lost.
Mutations, each planted alone and restored:
- the look leaves out the paths the batch names: the truncation test
fails (an empty output published 6.202167ms after the truncation)
- the re-baseline leaves out the paths held back: the truncation test
and the unit test fail (Some(""))
- the look leaves out the paths held back: the unit test fails, the
hold's deadline having moved
Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem being solved
mds-cli's output and watch layer crashed on closed pipes, leaked raw panic text and absolute paths, followed planted symlinks on writes and deletes, deleted hand-written files in directory watch, published empty outputs mid-save, and hid lint output bugs behind duplicated code paths.
Progress
wait_for_stderr_contains_strreturns the partial text on timeout, so count assertions built on it are vacuous #381); the cap test's skip is visible and the watch soak takes an allowlisted test input and counts skips separately (test(watch): cap test's cadence precondition fails CI on loaded runners #397)mds lintCLI per-file dispatch spine #173, refactor(lint-cli): lint.rs output-matrix ResultSink redesign #309)mds watch --debounce 0: an in-place editor save can compile the 0-byte truncate intermediate and leave a transiently empty output #380)Test Plan
Closes #157
Closes #389
Closes #390
Closes #160
Closes #309
Closes #173
Closes #380
Closes #381
Closes #397
Closes #256
Closes #257