feat(write): write xlsx through TabularWriter, with our own streaming zip writer - #81
Merged
Merged
Conversation
…tors, streamed deflated parts, zip64
…or formats that name sheets
… normalization turns them into spaces
… the xlsx row buffer cap from RowText's
…; docs: LibreOffice 1900 leap-year display note
main brought its own CRC-32 for gzip (TriasDev.Tabular.Archive.Crc32, with the ARM64 instruction). Two classes of the same name made the archive tests bind to the writer's; the zip writer now uses the archive one, and the writer's copy and its tests are gone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Closes #71. Part 2 of 5 of #67:
TabularWriter.Create(stream, TabularFormat.Xlsx)streams an xlsx workbook — several sheets, typed cells, flat memory — that the library's own import reads back as written and the Open XML SDK's schema validator accepts.Spec:
docs/superpowers/specs/2026-10-03-writing-design.md. Plan:docs/superpowers/plans/2026-10-03-writing-part-2-xlsx.md.ZipWriter, internal), becauseZipArchivedoes not fit: written to a stream that cannot seek, LibreOffice refuses its output; given one that can, it keeps each whole entry in memory. Small parts are stored with their sizes up front; each sheet streams throughDeflateStreamwith a data descriptor; zip64 past 4 GB; a slicing-by-8 CRC-32 of our own (System.IO.Hashingis a package).XlsxSheetWriter: inline strings (no shared-string table held in memory),_x000D_for CR and_x005F_before a literal_x, booleans, numbers, dates as serials across Excel's 1900 leap-year bug, four styles (General, date, date-time, and0for integers so an id does not show as1.23457E+11), column widths,ron every row and cell. It refuses what a workbook would not give back: a long a double cannot hold exactly or a decimal past 15 significant digits (write.precision-loss), a date before 1900 (write.date-out-of-range), text over 32,767 characters (write.text-too-long); more than 1,048,576 rows islimit.exceeded.[]:*?/\or control characters, no apostrophe at either end, notHistory, unique ignoring case), checked byTabularWriterfor every format that names sheets — ods inherits them.XlsxWriterOptions.CompressionLevel,Fastestby default for now; part 5 benchmarks it.XlsxCursor.TryFromSerialturned the rounded milliseconds into ticks with a double multiply, inexact past about year 2355 —9999-12-31 23:59:59.999read 24 µs short. It now multiplies inlong. Not benchmarked: it is one constant division and one long multiply per date cell, in place of one double multiply.RowText, the row buffer, is shared by the csv and xlsx writers (the csv writer's behaviour is unchanged; every csv test passes untouched). An abandoned writer now releases its open deflate entry.Performance (final review, Apple arm64, net10, 1,048,576 rows × 8 columns): 1.75 s at
Fastest, 1.3 s stored, 3.8 s atOptimal; 425 MB of sheet XML → a 76 MB file; about 3 MB allocated in total, no gen0 collections — memory is flat in the row count. Part 5 will measure buffered row writes (7–9 % measured) and a hardware CRC.Checked beyond the tests (final review): LibreOffice 25 headless converts a written workbook with every value correct; Info-ZIP, Python's
zipfileand streamingbsdtaraccept zip64 output; 5M random decimals and 20M random dates agree between writer and reader. Excel itself is checked by hand before the release.Tests: the Open XML SDK validator (test project only — the library still takes no package) on every workbook; round trips through
TabularImporterfor every value kind and edge value; a fuzz over three seeds; a full 1,048,576-row sheet read back to its last row and the next row refused; zip structures incl. zip64 at a lowered threshold; CRC against standard vectors and a bitwise reference; the reader fix against the old behaviour.Checklist
dotnet buildanddotnet testpass on net8.0 and net10.0 with zero warnings. (2324 tests)TryFromSerial; reasoned above, not benchmarked)ErrorCodesand the guide's error-code table agree. (no new code; the table names xlsx's causes)TabularFormat.Xlsxaccepted byCreate,TabularWriterOptions.Xlsx,XlsxWriterOptions)src/.