Repository navigation
fix(conformance): refuse a provisional tag that is not a §12 reading - #24
Merged
Merged
Conversation
The runner accepted any non-empty string as a case's provisional tag, so a tag naming a reading the owner has since settled passed silently. It now reads SPEC.md §12's list of provisional readings, the bullets after the §12 heading and before the next bold paragraph, with their line breaks folded to spaces, and refuses a case as malformed when its tag is not one of them. A SPEC.md with no §12 is no verdict. The membership check also rejects an empty or blank tag, so the separate non-empty test and its mutant go; a tag must still be a string. New mutants cover the check, the end of the list, the folding of continuation lines and the missing-§12 exit. Signed-off-by: Cameron Brooks <95648474+CameronBrooks11@users.noreply.github.com>
…nown-provisional-tag Signed-off-by: Cameron Brooks <95648474+CameronBrooks11@users.noreply.github.com>
The runner found SPEC.md's provisional list by the heading number "## 12.", so renumbering the sections would point it at another section and admit that section's bullets as readings. It now finds the one "## <n>. Provisional" heading by its title, and a SPEC.md with no such heading, or more than one, is no verdict. An empty "- " bullet no longer counts as a reading, and the check that a tag is a non-empty string is back, so an empty tag is refused whatever SPEC.md holds. New tests cover an empty bullet, a stripped bullet, the list ending at the next heading, a renumbered section and a doubled section, and call load_expect directly so the non-empty test checks what its name says. Each has a mutant in the mutation gate. Signed-off-by: Cameron Brooks <95648474+CameronBrooks11@users.noreply.github.com>
The runner reads SPEC.md §12's provisional list only as plain "- " bullets with two-space continuation lines, ending at the first bold paragraph or heading. Say so beside the provisional tag, so an edit to §12 keeps the list in a shape the runner reads. Signed-off-by: Cameron Brooks <95648474+CameronBrooks11@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
conformance/run_cases.pyaccepted any non-empty string as a case'sprovisionaltag. A tag naming a reading the owner has since settled passed silently. With"provisional": "Ids are ASCII letters, digits and_(§4.4)."added togrammar/ids-ascii-forms(a settled reading), the old runner gives322 passed … rc=0. This runner gives321 passed, 1 malformed … rc=2.Change
## <n>. Provisional…section (§12). The heading is found by its title, not its number. A SPEC.md with no such heading, or more than one, is no verdict (exit 2).-bullet after that heading and before the next bold paragraph (**Settled by the owner…) or heading. Each bullet is stripped, and its two-space continuation lines are folded in with single spaces.conformance/cases/README.mdnow says the list must keep that shape: plain-bullets with two-space continuation lines. Other shapes (a bold lead-in, a###subheading,*bullets) give an empty list, which fails closed. Lazily wrapped or nested bullets would be misread, which is why the README states the constraint.Tests
New tests in
tests/test_run_cases.py:-bullet;## 12.);load_expectrefusing a non-string or blank tag even where the readings would admit it.Each test that guards a fix failed before that fix:
assert 0 == 2;The stripping and heading-stop tests exist to kill mutants that used to survive.
Mutants added to
just mutants, all killed:runner-provisional-tag-uncheckedrunner-settled-list-read-as-provisionalrunner-list-ends-only-at-boldrunner-provisional-bullet-not-foldedrunner-provisional-continuation-droppedrunner-provisional-bullet-not-strippedrunner-empty-bullet-is-a-readingrunner-provisional-heading-by-numberrunner-two-provisional-sections-allowedrunner-no-section-12-is-a-passrunner-provisional-any-string(restored)Gates (head of branch, after merging main)