Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a readable stream supplies CSV in separate chunks, a
toorto_linelimit can close the parser's writable side while the source is still sending data. This raisesERR_STREAM_WRITE_AFTER_ENDinstead of completing after the requested records.Wait for the final records to be consumed before ending the writable side, then wait for it to finish before destroying the parser. Hold the transform callback while those records are waiting so a paused consumer does not make the parser drain and discard the rest of the input. Buffered write and flush callbacks can then finish normally.
The regression tests cover separate input chunks, async iteration and a consumer that pauses after each record. The paused-consumer cases also check that a 1,000-row source stops early rather than being read to the end. The previously skipped
Readablecase is enabled.Testing: the 12 focused tests pass on Node 20.20.2, 22.23.3 and 24.13.1 on Windows; five fail on the original source. TypeScript and changed-file ESLint pass. The broader parser suite passes 549 tests with two existing performance tests pending, excluding the sample loader because its unchanged file-URL path handling fails on Windows. The existing npm test script also needs a direct Mocha invocation because of its single-quoted glob.
There is still a separate limit:
pipeline()with an asynchronous upstream source can reportERR_STREAM_PREMATURE_CLOSEwhen truncating input. This also occurs on the original source. Other operating systems, browser execution and generated bundles were not tested.Refs #333 and #410.