Repository navigation
cli: detect output filename collisions before processing to prevent data loss - #4870
siddiqueirshad wants to merge 1 commit into
Conversation
… processing
When using --output-dir-flat, FIO_checkFilenameCollisions() was previously invoked only after the compression/decompression loop had finished. Consequently, when two inputs mapped to the same output name with --rm, the second file overwrote the first and both sources were deleted, causing silent data loss.
Additionally on Windows:
- File paths containing forward slashes were not split because only PATH_SEP ('\') was checked.
- Filename collisions were checked case-sensitively, failing to catch colliding names like 'a.txt' and 'A.txt' on case-insensitive filesystems.
This change:
1. Runs FIO_checkFilenameCollisions() prior to compressing or decompressing files, returning an error status (1) to abort processing if colliding output names are detected.
2. Checks both PATH_SEP and '/' on Windows when extracting filenames, aligning with FIO_createFilename_fromOutDir().
3. Uses case-insensitive comparison on Windows (_stricmp) during sort and deduplication.
Fixes meta#4854
Signed-off-by: siddiqui irshad <mohdirshad1306@gmail.com>
|
Hi @siddiqueirshad! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
Description of changes
Fixes #4854.
When
--output-dir-flatwas specified,FIO_checkFilenameCollisions()was previously executed only after the entire compression or decompression loop had finished. If multiple inputs produced colliding output filenames (such asd1/x.txtandd2/x.txtmapping toout/x.txt.zst), the second compressed file overwrote the first, and with--rmboth source files were deleted, resulting in silent data loss with exit code 0.Additionally on Windows:
FIO_checkFilenameCollisions()split paths using onlyPATH_SEP(\\). Input paths written with forward slashes (/) were not separated from directory components, bypassing collision detection.a.txtandA.txton case-insensitive filesystems.Changes Made
FIO_checkFilenameCollisions()outputs an error and returns non-zero, causingFIO_compressMultipleFilenames/FIO_decompressMultipleFilenamesto abort immediately with exit code 1 before any file is created, overwritten, or deleted.extractFilename(filename, '/')on Windows to correctly parse names when paths use forward slashes (matchingFIO_createFilename_fromOutDir())._stricmpfor sorting and adjacent comparison on Windows so that case variations are recognized as colliding filenames.Fixes #4854