Conversation
Adds a small xUnit project (Tests/ColorCode.Core.UnitTests) covering the JSON language's Regex_String. Partial_json_does_not_catastrophically_backtrack feeds an array with an unterminated string value and asserts highlighting finishes within a time budget; against current main it does not (O(2^n) backtracking, issue CommunityToolkit#45), so this commit fails and the following fix commit makes it pass. Valid_json_still_highlights_keys_strings_and_escapes pins the tokenisation of valid JSON so the fix can be shown to change nothing for good input. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: SandRock <sandrock@sandrock.fr>
…olkit#45) The string pattern used Friedl's problem shape: the repeating group (?:\[^\r\n]|[^"\\]*)* has an alternative ([^"\\]*) that can match the empty string, so when a match ultimately fails (e.g. a string value in an array, matched by the key rule but with no ':' following) the engine backtracks through every way to partition the string body — O(2^n) states. A 25-character unterminated value already exceeds seconds; longer values hang indefinitely. Replace it with Friedl's unrolled loop, "[^"\\]*(?:\\.[^"\\]*)*", which removes the ambiguity: linear time, and it matches valid strings (including escaped quotes and backslashes) identically to before. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: SandRock <sandrock@sandrock.fr>
Author
|
@dotnet-policy-service agree |
This was referenced Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix catastrophic regex backtracking in the JSON parser
The JSON language's string pattern in
ColorCode.Core/Compilation/Languages/Json.csbacktracks catastrophically (O(2ⁿ)) on malformed/partial JSON, freezing the highlighter. This is the root cause behind both #45 (the diagnosis, on JSON arrays) and #36 (JSON parsing "deadlock/stuck… without any Exception" — same array-of-strings trigger).The bug
The repeating group has an alternative —
[^"\\]*— that can match the empty string. When a match ultimately fails (e.g. the key rule[,{]\s*("…")\s*:matches a string value in an array but finds no:after it), the engine backtracks through every way to partition the string body between the outer and inner[^"\\]*, i.e. O(2ⁿ) states. A 25-character unterminated value already runs for seconds; longer ones hang indefinitely.The fix
One line — Friedl's unrolled loop, which removes the empty-matchable ambiguity:
Verification
Matching an unterminated string of n ordinary characters:
And the new pattern matches valid strings identically to the old one —
"John Doe","", escaped quote"a\"b", escaped backslash"C:\\path","line1\nstill"all tokenize the same — so highlighting of well-formed JSON is unchanged.Commits
Two commits so the change is easy to verify:
Tests/ColorCode.Core.UnitTests). The regression test fails at this commit (backtracking) and the correctness test passes.Relative to the earlier #43 (which added a word-boundary tweak and is still open), this uses the canonical unrolled-loop form and includes tests plus a proof of identical behavior on valid input.
It also introduces the repository's first xUnit test project (
Tests/ColorCode.Core.UnitTests) — a small step toward #13.Fixes #45.
Fixes #36.
Signed off (DCO).
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com