Skip to content

fix(csv-parse): finish limited streams without write-after-end errors - #513

Open
x0Lazarus wants to merge 1 commit into
adaltas:masterfrom
x0Lazarus:fix/parse-stream-limit-finish
Open

x0Lazarus wants to merge 1 commit into
adaltas:masterfrom
x0Lazarus:fix/parse-stream-limit-finish

Conversation

@x0Lazarus

Copy link
Copy Markdown
Contributor

When a readable stream supplies CSV in separate chunks, a to or to_line limit can close the parser's writable side while the source is still sending data. This raises ERR_STREAM_WRITE_AFTER_END instead of completing after the requested records.

Wait for the final records to be consumed before ending the writable side, then wait for it to finish before destroying the parser. Hold the transform callback while those records are waiting so a paused consumer does not make the parser drain and discard the rest of the input. Buffered write and flush callbacks can then finish normally.

The regression tests cover separate input chunks, async iteration and a consumer that pauses after each record. The paused-consumer cases also check that a 1,000-row source stops early rather than being read to the end. The previously skipped Readable case is enabled.

Testing: the 12 focused tests pass on Node 20.20.2, 22.23.3 and 24.13.1 on Windows; five fail on the original source. TypeScript and changed-file ESLint pass. The broader parser suite passes 549 tests with two existing performance tests pending, excluding the sample loader because its unchanged file-URL path handling fails on Windows. The existing npm test script also needs a direct Mocha invocation because of its single-quoted glob.

There is still a separate limit: pipeline() with an asynchronous upstream source can report ERR_STREAM_PREMATURE_CLOSE when truncating input. This also occurs on the original source. Other operating systems, browser execution and generated bundles were not tested.

Refs #333 and #410.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant