diff --git a/docs/compatibility.md b/docs/compatibility.md index abe0a8e6e..ab7c55bd2 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -42,6 +42,7 @@ only the built-in decoders in `reader`; no encoder class is loaded. | Arrow extension array import affecting Variant shape | Rust 0.74.0 (#8125) | Untested against the currently pinned v0.88.0 fixtures; #8125 not yet re-verified. | | `vortex.dict` **layout** over a values pool that is neither VarBin- nor primitive-shaped (e.g. a dict-encoded `vortex.uuid`, whose storage is `FixedSizeList(U8, 16)`) | Not written by Rust: its dict layout admits only `Primitive \| Utf8 \| Binary` (`dict_layout_supported`), and its dict compressor schemes only integers, floats and strings | ⚠️ Unreachable from Rust- or Java-written files. Every type vortex-jni writes reads back exactly, dict-encoded or not (`DictAllTypesInteropIntegrationTest`, which also fails if a Rust bump starts dict-encoding another type). A foreign file with such a pool fails with `VortexException("unsupported dict values shape: …")`. | | Duplicate struct field names | Rust writer rejects ("StructLayout must have unique field names"); Rust reader tolerates foreign files (first-match access) | ⚠️ Deliberate divergence on read: Java rejects such files with `VortexException("duplicate field name in file schema")` instead of tolerating them — the name-keyed `Chunk` API cannot represent both columns, and silent column loss is worse than a loud failure on a file the reference writer refuses to produce. Java's writer mirrors the Rust writer's rejection. | +| Decimal values beyond their declared precision | Wire-legal (a decimal array stores whatever its unscaled integers hold). Rust 0.90's Arrow import keeps such values; Rust's development branch rejects them ([vortex-data/vortex#10897](https://github.com/vortex-data/vortex/pull/10897): "Arrow decimal import requires every valid value to be within the precision") | ⚠️ Deliberate strictness on write: vortex-java's writer refuses a value with more digits than its column's precision, or more fractional digits than its scale (`IllegalArgumentException` naming the column and row, pinned by `ChunkImplTest`), whether the value comes through the API or a Parquet import. A value outside its declared type is wrong data, not a wider decimal. | | Blank / control-character field names | Wire-legal; the Rust writer produces `""` and whitespace-only names. NUL (`U+0000`) additionally aborts the Rust toolchain: Arrow FFI schema export hits a panic-cannot-unwind in `arrow-rs` (`ffi_stream::get_schema`) and SIGABRTs the process (measured against vortex-jni 0.75.0) | ⚠️ Deliberate strictness BOTH ways: vortex-java's writer refuses blank and control-character field names (`IllegalArgumentException`), and its reader rejects files carrying them (`VortexException` naming the producing pipeline as the likely bug) — the JSON-`""`-key principle: wire-legal is a floor, not a policy. Printable names of any shape (`$`-runs, spaces inside, emoji) are legal and round-trip intact both directions (measured; pinned by `ColumnNameEdgeCasesIntegrationTest`). | ## SIMD kernels and the Vector API