fix(share): non-destructive multi-reader getData - #53
Closed
davidbudzynski wants to merge 4 commits into
Closed
Conversation
Reads no longer shm_unlink, so N processes (e.g. mirai_map daemons) can attach while the owner handle from shareData() stays alive. Lifetime is owned by clearData()/finalizer. Also strdup shm names (was dangling CHAR pointer), always ftruncate on re-share, and fix error-path leaks. No new dependencies.
- Length segment now carries a {len, gen} seqlock header (len stays at
offset 0 for back-compat). Readers fail loudly ('please retry')
instead of risking torn data when racing a concurrent writer.
- Add clearShared(map_name) unlink-by-name for orphan recovery
(e.g. after a crashed session); no-op on Windows where mappings
die with the last handle.
- Same-size re-share republishes in place; size change recreates
objects (macOS rejects ftruncate on re-opened shm fds).
- fstat-clamped data mapping avoids SIGBUS on concurrent shrink.
- No new dependencies.
…rtable tests - Validate MapLengthName everywhere; EXTPTRSXP check in clearData; RAWSXP check + XLENGTH in creator; R-side map_name/verbose checks. - Ordered header access via __atomic load/store (acquire/release); 32-bit gen for single-copy atomicity; loud fallback warning. - Busy-name detection: odd generation fails loudly (dual-writer or crashed writer) instead of interleaving; recover via clearShared. - Reader opens header before payload so resize mixes are unreachable; fstat header before mapping (SIGBUS guard); early fail-fast check. - Unlink-on-error only for objects this call created (fresh-tracked). - Exact .Call arities; DWORD size split on Windows; errno saved before diagnostics; SHM_META_SIZE used consistently. - Tests hermetic via pre-clean; Windows branches for clearShared; POSIX-only no-clear re-share block; pkgdown topic updated.
…open - The fstat guard made the POSIX meta-map block self-contained, orphaning the shared closing brace of WIN32's NULL check, which broke the Windows build (nested-function cascade). Shared-tail structure restored. - shm_open() needs its mode argument even without O_CREAT: glibc declares three parameters (macOS tolerates two). Fixes Linux build. - Guard shm_dup_string to POSIX (unused-function warning on Windows would fail error_on=warning CI).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #43
getData()used to unlink the shared memory on first read, so only one process could ever read it. That defeats the point when you want to share a big object once and have several workers (e.g.mirai_mapdaemons) pick it up while the main session stays alive.What this does, with no new dependencies:
getData()no longer destroys the segment. Read as many times as you want, from any process as the same user. The owner handle fromshareData()owns the lifetime now — keep it alive andclearData()it when everyone is done.please retry) instead of silently returning mixed bytes. One writer per name; concurrent writers must use unique names.clearShared(map_name)to remove a segment by name, e.g. leftovers after a crash. No-op on Windows (mappings die with the last handle there).ftruncateon re-opened shm, so it recreates instead). On Windows clear a live name before growing it.Heads-up on behavior change: code that relied on read-to-clean (
share→getwith noclearData) will now leak the segment until the owner handle is cleared/GC'd. The documented pattern (share→ read →clearData) is unchanged.Checked with repeated/multi-process reads, parallel readers, cross-process resize, a 12k-read race stress (0 torn reads),
kill -9orphan recovery, andR CMD check(tests OK, no new warnings).