Repository navigation
refactor: replace the vendored zip and Vorbis code with own implementations - #2950
Merged
Merged
Conversation
…ct in C# and Kotlin The CRC-32 table and state used unsigned 32-bit values and >>>. In the transpiled C# and Kotlin code these hit out-of-range double-to-int conversions and a signed shift, so exported zip entries carried wrong checksums there. Add known-answer tests.
…lementation Add DeflateFormat with the parts of the format shared by encoder and decoder (length/distance tables, code length order, fixed code, canonical code assignment) and its integer rules for the C# and Kotlin builds. The new Inflater decodes a whole stream into one growing buffer using single-level lookup tables, reads input bytes only when needed so it stops exactly at the end of the stream, and validates the data as strictly as zlib. It replaces the Haxe based Inflate, HuffTools and Huffman.
…lementation The new Deflater compresses a whole buffer at once: hash chains with lazy matching using zlib's default search limits, and per block the smallest of stored, fixed and dynamic encoding. Dynamic codes are length-limited Huffman codes computed with the Moffat-Katajainen algorithm. All results are deterministic and identical on all platforms. ZipWriter now computes the CRC-32 itself. The SharpZipLib based Deflater, DeflaterEngine, DeflaterHuffman, DeflaterConstants, PendingBuffer and the unused Adler32 are removed. Tests cover round trips, golden output, the code length limiting and, on the web target, interoperability with zlib.
The zip inflate and deflate algorithms are no longer based on these libraries.
Computes the Vorbis IMDCT through a DCT-IV based on an n/4 point complex FFT. Covers all Vorbis block sizes (64-8192), including 64 and 128 where the current implementation produces wrong output.
…ementation Decodes the Ogg Vorbis samples of SF3 soundfonts with a decoder written from the Vorbis I specification, replacing the NVorbis port: - Codewords at the end of a packet are decoded when they fit into the remaining bits. They were dropped before, which left an error of up to -36 dBFS compared to libvorbis in SF3 samples. - The output is cut at the final granule position in all streams. - Packets spanning Ogg pages are read correctly. - Parsed setups are cached per Hydra: SF3 soundfonts share a few setups across all samples. - Floor type 0 (only used by encoders before 2002) is not supported. The tests now compare against libvorbis output instead of the previous decoder's output.
Short-paged.ogg is Short.ogg with every packet split across pages (valid CRCs, decoded identically by libvorbis). Mutated and truncated streams must either decode or fail with a FormatError, which guards the C# and Kotlin ports against out-of-range access.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
AI-authored disclosure (
alphatab-ai-authored-v1)Portions of this content were authored by an AI agent. The agent has read
AGENTS.md and the human submitter accepts responsibility for
compliance with the rules in that document.
Issues
Fixes #2948
Proposed changes
alphaTab shipped ports of three libraries for its low-level algorithms:
This PR replaces them with alphaTab-owned implementations written from RFC 1951 and the Vorbis I specification, plus standard published algorithms (Moffat–Katajainen code lengths, a DCT-IV based IMDCT). The four library notices are removed from
LICENSE.header. zlib and libvorbis are only used as references in tests. TinySoundFont stays, as planned in the issue.The commits are split per step: zip (CRC-32 fix, inflater, deflater, licence), then Vorbis (IMDCT, decoder, robustness tests, licence).
Zip (
src/zip)DeflateFormat: the parts shared by both directions. These are the length/distance tables, the code length order, the fixed code and the canonical code assignment.Inflater(replacesInflate,HuffTools,Huffman):Deflater(replacesDeflater,DeflaterEngine,DeflaterHuffman,DeflaterConstants,PendingBufferand the unusedAdler32):ZipWritercomputes the CRC-32 itself.Vorbis (
src/synth/vorbis)OggReader: concatenates the page bodies of the first logical stream. Packet boundaries come from the lacing values alone, so packets spanning pages need no special handling.VorbisBitReader: reads least significant bit first. Reads past the end of a packet return zero bits and set an end-of-packet flag. A codeword that fits into the remaining bits still decodes.VorbisCodebook:VorbisFloor(floor 1),VorbisResidue(types 0/1/2),VorbisSetup(setup header, mappings, modes).Mdct: the inverse MDCT as a DCT-IV over an n/4-point complex FFT. On its own it is 1.5–2× faster than the previous unrolled implementation.VorbisDecoder:Hydrakeeps one instance, so SF3 samples sharing a setup parse it once (default.sf3: 182 samples, 3 setups);Hydrapasses the byte range ofsampleDatainstead of slicing a copy.Defects of the previous code that are fixed
>>>. C# and Kotlin convert these wrongly.123456789came out as-2139127681instead of-873187034.NaNaudio in JS, exceptions other thanFormatErrorin C#/Kotlin.Behaviour changes
FormatError. Before, they threw aTypeErroror anAlphaTabError.FormatError). Only encoders before 2002 used it, no SF3 tool writes it, and stb_vorbis leaves it out as well.C# and Kotlin
Both rewrites follow integer rules documented in
DeflateFormatandVorbisBitReader:>>>and no compound bit assignments;subarray;Building locally: run
npm run clean --workspace=packages/csharpandnpm run clean --workspace=packages/kotlinfirst. Stale generated files of the deleted classes otherwise collide.Tests and verification
test/zip/Deflate.test.ts):test/zip/DeflateInterop.test.ts, web only): zlib inflates our output for the whole test-data zip corpus, and we inflate zlib's output in all its modes.Short.ogg,Example.oggand three default.sf3 samples (one per block size pair: 512/1024, 512/512, 256/2048) must have the exact libvorbis length and a maximum difference below 1e-5;*_libvorbis.pcmreferences were produced with libvorbisfile (float output);*_alphaTab.pcmreferences, which held the previous decoder's output including the end-of-packet defect.Short-paged.ogg(every packet split across pages, valid CRCs, decoded identically by libvorbis) decodes bit-identically toShort.ogg.Short.oggeither decode or throwFormatError.Zip results (the 1,649 zip entries of the test data, 19.7 MB uncompressed):
Vorbis speed (Node 24, decoding all samples of a soundfont):
Size (minified / gzip):
Found along the way, not changed here
Int32Array.Setcopies to the wrong offset when the destination is asubarrayview. The new code doesn't use it.GpifWriter:frets.sort()sorts numbers without a comparator. JS sorts them as text, C#/Kotlin numerically, so the platforms can write different output.@recordfield name (e.g.long) produces an unescaped named argument;(a >> 2) + (b >> 2)asInt.Checklist
AI authorship disclosure
(
alphatab-ai-authored-v1) is present at the top of this body, and I havepersonally reviewed every change and can explain each one
Further details