Skip to content

fix: preserve trailing whitespace in partial JSON strings - #23

Open
Lushiyao5 wants to merge 2 commits into
promplate:mainfrom
Lushiyao5:fix/preserve-partial-string-trailing-spaces
Open

Lushiyao5 wants to merge 2 commits into
promplate:mainfrom
Lushiyao5:fix/preserve-partial-string-trailing-spaces

Conversation

@Lushiyao5

Copy link
Copy Markdown

An unfinished JSON string loses trailing spaces before completion. For example, loads('\"hello ') returns 'hello' instead of 'hello '. The same loss affects strings inside arrays and objects, on both completion paths.

Use lstrip() rather than strip() before completion. Add 28 regression/control cases for top-level and nested strings, Unicode whitespace, completed values, and partial-string restrictions. The cases live in tests/test_examples.py, which CI runs.

Validation on Windows, Python 3.12.10:

  • 18 of the 28 new cases fail against upstream e0aa5e2; all 28 pass with this change.
  • 38 example/regression tests, 2 existing property tests, and 2 performance test functions pass.
  • Black and isort pass on both changed files; Pyright reports 0 errors across src/ using the project virtual environment and the playground extra.

Related: #22 fixes a separate leading-whitespace offset error in the same function. Its current patch still calls strip(). This change addresses trailing string content; the patches may need coordination depending on merge order.

AI assistance: OpenAI Codex helped investigate, implement, and test this change. The reproduction and verification logs are available for review.

@bolt-new-by-stackblitz

Copy link
Copy Markdown

Review PR in StackBlitz Codeflow Run & review this pull request in StackBlitz Codeflow.

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0b2fa269-497a-400a-a9a5-212b37506696
📥 Commits

Reviewing files that changed from the base of the PR and between 9c1af26 and 2556d52.

📒 Files selected for processing: 2
  • src/partial_json_parser/core/complete.py
  • tests/test_examples.py

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

📝 Summary

Preserve trailing whitespace in unfinished JSON strings by removing only leading whitespace before completion.

Add regression tests for top-level and nested strings, both completion paths, Unicode whitespace, and partial-string restrictions. The PR reports that all 28 new cases pass.

_fix now removes leading whitespace without removing trailing whitespace before parsing. complete_any adjusts number and literal parsing and passes the top-level flag to numeric completion. Incomplete-string recovery uses a trimmed endpoint when handling escapes and control characters. Parameterized tests cover whitespace and recovery behavior for strings, literals, and numbers in both fixing modes.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 2556d

This change preserves trailing spaces in partial JSON strings. No concrete merge-blocking risk was found in the supplied context.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description check Passed The description clearly explains the trailing-whitespace bug, the implementation change, regression coverage, and validation results.
Title check Passed The title concisely and accurately identifies the primary change: preserving trailing whitespace in partial JSON strings.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 4 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="src/partial_json_parser/core/complete.py" line_range="21" />
<code_context>
     try:
-        result = complete_any(json_string.strip(), allow, is_top_level)
+        # Trailing whitespace can belong to an unfinished string.
+        result = complete_any(json_string.lstrip(), allow, is_top_level)
         if result is False:
             raise PartialJSON
</code_context>
<issue_to_address>
**Partial literals stop completing**

When a supported partial literal is followed by JSON whitespace, `complete_any` passes the whitespace to its literal-prefix checks, which reject the prefix; `parse_json` then raises `MalformedJSON` instead of completing the literal, including when it appears inside a collection or is a special value.

Make literal-prefix checks ignore trailing JSON whitespace while retaining whitespace that belongs to unfinished strings.
</issue_to_address>

### Comment 2
<location path="src/partial_json_parser/core/complete.py" line_range="21" />
<code_context>
     try:
-        result = complete_any(json_string.strip(), allow, is_top_level)
+        # Trailing whitespace can belong to an unfinished string.
+        result = complete_any(json_string.lstrip(), allow, is_top_level)
         if result is False:
             raise PartialJSON
</code_context>
<issue_to_address>
**Leading whitespace truncates strings**

When valid leading JSON whitespace precedes an unfinished string, `complete_any` returns an index relative to the `lstrip()` result, but `_fix` applies it to the original `json_string`; the returned head loses part of the string and callers receive a truncated value.

Adjust the returned index for the stripped leading whitespace before `_fix` slices the original string.
</issue_to_address>

### Comment 3
<location path="src/partial_json_parser/core/complete.py" line_range="21" />
<code_context>
     try:
-        result = complete_any(json_string.strip(), allow, is_top_level)
+        # Trailing whitespace can belong to an unfinished string.
+        result = complete_any(json_string.lstrip(), allow, is_top_level)
         if result is False:
             raise PartialJSON
</code_context>
<issue_to_address>
**Unfinished strings fail parsing**

When an unfinished string ends with an unescaped tab or newline, or a backslash followed by whitespace, `_fix` preserves the invalid suffix and `complete_str` appends a quote without validating it; `parse_json` passes invalid JSON to its parser, which raises `JSONDecodeError` instead of returning a completed value.

Handle invalid trailing whitespace and dangling escapes in `complete_str` so closing the string produces valid JSON.
</issue_to_address>

### Comment 4
<location path="src/partial_json_parser/core/complete.py" line_range="21" />
<code_context>
     try:
-        result = complete_any(json_string.strip(), allow, is_top_level)
+        # Trailing whitespace can belong to an unfinished string.
+        result = complete_any(json_string.lstrip(), allow, is_top_level)
         if result is False:
             raise PartialJSON
</code_context>
<issue_to_address>
**Trailing spaces break partial escapes**

When an unfinished string ends with a truncated `\u`, `\U`, or `\x` escape followed by whitespace, `complete_str` searches for an incomplete `\u`, `\U`, or `\x` escape only within a short suffix. Retained spaces can push the escape outside that search, so it appends a quote after invalid JSON and parsing fails; when the escape is still found, it truncates the escape and the following spaces too.

Handle the incomplete escape separately from subsequent string content so recovery does not discard trailing spaces or emit an invalid escape.
</issue_to_address>

Sourcery assessment

Approval pending. 4 findings to address first.

Blocking findings: src/partial_json_parser/core/complete.py:21, src/partial_json_parser/core/complete.py:21, src/partial_json_parser/core/complete.py:21, src/partial_json_parser/core/complete.py:21


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

try:
result = complete_any(json_string.strip(), allow, is_top_level)
# Trailing whitespace can belong to an unfinished string.
result = complete_any(json_string.lstrip(), allow, is_top_level)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High · Leading whitespace truncates strings

When valid leading JSON whitespace precedes an unfinished string, complete_any returns an index relative to the lstrip() result, but _fix applies it to the original json_string; the returned head loses part of the string and callers receive a truncated value.

Adjust the returned index for the stripped leading whitespace before _fix slices the original string.

Prompt for AI agents
In `src/partial_json_parser/core/complete.py` at line 21:

**Leading whitespace truncates strings**

When valid leading JSON whitespace precedes an unfinished string, `complete_any` returns an index relative to the `lstrip()` result, but `_fix` applies it to the original `json_string`; the returned head loses part of the string and callers receive a truncated value.

Adjust the returned index for the stripped leading whitespace before `_fix` slices the original string.

try:
result = complete_any(json_string.strip(), allow, is_top_level)
# Trailing whitespace can belong to an unfinished string.
result = complete_any(json_string.lstrip(), allow, is_top_level)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Unfinished strings fail parsing

When an unfinished string ends with an unescaped tab or newline, or a backslash followed by whitespace, _fix preserves the invalid suffix and complete_str appends a quote without validating it; parse_json passes invalid JSON to its parser, which raises JSONDecodeError instead of returning a completed value.

Handle invalid trailing whitespace and dangling escapes in complete_str so closing the string produces valid JSON.

Prompt for AI agents
In `src/partial_json_parser/core/complete.py` at line 21:

**Unfinished strings fail parsing**

When an unfinished string ends with an unescaped tab or newline, or a backslash followed by whitespace, `_fix` preserves the invalid suffix and `complete_str` appends a quote without validating it; `parse_json` passes invalid JSON to its parser, which raises `JSONDecodeError` instead of returning a completed value.

Handle invalid trailing whitespace and dangling escapes in `complete_str` so closing the string produces valid JSON.

try:
result = complete_any(json_string.strip(), allow, is_top_level)
# Trailing whitespace can belong to an unfinished string.
result = complete_any(json_string.lstrip(), allow, is_top_level)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Trailing spaces break partial escapes

When an unfinished string ends with a truncated \u, \U, or \x escape followed by whitespace, complete_str searches for an incomplete \u, \U, or \x escape only within a short suffix. Retained spaces can push the escape outside that search, so it appends a quote after invalid JSON and parsing fails; when the escape is still found, it truncates the escape and the following spaces too.

Handle the incomplete escape separately from subsequent string content so recovery does not discard trailing spaces or emit an invalid escape.

Prompt for AI agents
In `src/partial_json_parser/core/complete.py` at line 21:

**Trailing spaces break partial escapes**

When an unfinished string ends with a truncated `\u`, `\U`, or `\x` escape followed by whitespace, `complete_str` searches for an incomplete `\u`, `\U`, or `\x` escape only within a short suffix. Retained spaces can push the escape outside that search, so it appends a quote after invalid JSON and parsing fails; when the escape is still found, it truncates the escape and the following spaces too.

Handle the incomplete escape separately from subsequent string content so recovery does not discard trailing spaces or emit an invalid escape.

Copy link
Copy Markdown
Author

I compared these findings against upstream e0aa5e2 and the initial PR commit 9c1af26.

The follow-up restores upstream recovery for partial literals, trailing control whitespace, and incomplete escapes, while retaining valid trailing string whitespace. It also preserves the existing numeric Allow restrictions.

The leading-whitespace offset issue predates this PR: upstream also truncates unfinished strings with a leading prefix. Open PR #22 already addresses that issue, and this PR's description notes the overlap. I have kept that correction separate to avoid duplicating the existing work.

Local verification on Windows / Python 3.12.10: 148 tests pass, including the property and performance test functions. All 94 compatibility cases also pass against the original upstream implementation. Black, isort, and full-source Pyright checks pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant