Skip to content

feat(write): write xlsx through TabularWriter, with our own streaming zip writer - #81

Merged
vaceslav merged 16 commits into
mainfrom
feat/write-xlsx
Oct 3, 2026
Merged

vaceslav merged 16 commits into
mainfrom
feat/write-xlsx

Conversation

@vaceslav

@vaceslav vaceslav commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

What and why

Closes #71. Part 2 of 5 of #67: TabularWriter.Create(stream, TabularFormat.Xlsx) streams an xlsx workbook — several sheets, typed cells, flat memory — that the library's own import reads back as written and the Open XML SDK's schema validator accepts.

Spec: docs/superpowers/specs/2026-10-03-writing-design.md. Plan: docs/superpowers/plans/2026-10-03-writing-part-2-xlsx.md.

  • Our own zip writer (ZipWriter, internal), because ZipArchive does not fit: written to a stream that cannot seek, LibreOffice refuses its output; given one that can, it keeps each whole entry in memory. Small parts are stored with their sizes up front; each sheet streams through DeflateStream with a data descriptor; zip64 past 4 GB; a slicing-by-8 CRC-32 of our own (System.IO.Hashing is a package).
  • XlsxSheetWriter: inline strings (no shared-string table held in memory), _x000D_ for CR and _x005F_ before a literal _x, booleans, numbers, dates as serials across Excel's 1900 leap-year bug, four styles (General, date, date-time, and 0 for integers so an id does not show as 1.23457E+11), column widths, r on every row and cell. It refuses what a workbook would not give back: a long a double cannot hold exactly or a decimal past 15 significant digits (write.precision-loss), a date before 1900 (write.date-out-of-range), text over 32,767 characters (write.text-too-long); more than 1,048,576 rows is limit.exceeded.
  • Sheet names follow Excel's rules (1–31 characters, none of []:*?/\ or control characters, no apostrophe at either end, not History, unique ignoring case), checked by TabularWriter for every format that names sheets — ods inherits them.
  • XlsxWriterOptions.CompressionLevel, Fastest by default for now; part 5 benchmarks it.
  • Reader fix: XlsxCursor.TryFromSerial turned the rounded milliseconds into ticks with a double multiply, inexact past about year 2355 — 9999-12-31 23:59:59.999 read 24 µs short. It now multiplies in long. Not benchmarked: it is one constant division and one long multiply per date cell, in place of one double multiply.
  • RowText, the row buffer, is shared by the csv and xlsx writers (the csv writer's behaviour is unchanged; every csv test passes untouched). An abandoned writer now releases its open deflate entry.

Performance (final review, Apple arm64, net10, 1,048,576 rows × 8 columns): 1.75 s at Fastest, 1.3 s stored, 3.8 s at Optimal; 425 MB of sheet XML → a 76 MB file; about 3 MB allocated in total, no gen0 collections — memory is flat in the row count. Part 5 will measure buffered row writes (7–9 % measured) and a hardware CRC.

Checked beyond the tests (final review): LibreOffice 25 headless converts a written workbook with every value correct; Info-ZIP, Python's zipfile and streaming bsdtar accept zip64 output; 5M random decimals and 20M random dates agree between writer and reader. Excel itself is checked by hand before the release.

Tests: the Open XML SDK validator (test project only — the library still takes no package) on every workbook; round trips through TabularImporter for every value kind and edge value; a fuzz over three seeds; a full 1,048,576-row sheet read back to its last row and the next row refused; zip structures incl. zip64 at a lowered threshold; CRC against standard vectors and a bitwise reference; the reader fix against the old behaviour.

Checklist

  • A test that failed before the change and passes after it (for a fix or a new behaviour).
  • dotnet build and dotnet test pass on net8.0 and net10.0 with zero warnings. (2324 tests)
  • If the read path changed: measured, and the numbers are in the description. (one arithmetic change in TryFromSerial; reasoned above, not benchmarked)
  • If an error code was added or changed: ErrorCodes and the guide's error-code table agree. (no new code; the table names xlsx's causes)
  • Public API changes are described, and breaking ones are called out. (additive: TabularFormat.Xlsx accepted by Create, TabularWriterOptions.Xlsx, XlsxWriterOptions)
  • No third-party package in src/.

…; docs: LibreOffice 1900 leap-year display note
main brought its own CRC-32 for gzip (TriasDev.Tabular.Archive.Crc32, with the ARM64 instruction).
Two classes of the same name made the archive tests bind to the writer's; the zip writer now uses
the archive one, and the writer's copy and its tests are gone.
@vaceslav
vaceslav merged commit 6451e59 into main Oct 3, 2026
9 checks passed
@vaceslav
vaceslav deleted the feat/write-xlsx branch October 3, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Writing part 2: own zip writer, CRC-32 and xlsx

1 participant