Skip to content

fix(cli): --output-dir-flat collisions with --rm delete a source whose output was overwritten #623

Closed

Description

@polaz

Problem

With --output-dir-flat, two inputs whose outputs land on the same name in the flat directory overwrite each other. With -f --rm both sources are then removed and the run exits 0, so the first input is lost. The collision is reported only after every input has been processed, as a warning at display level 2.

On macOS (and Windows) the file system ignores case, while the collision check compares names byte for byte: d1/a.txt and d2/A.txt write one out/a.txt.zst with no warning at all.

Reproduced on main on throwaway copies:

mkdir d1 d2 out
printf 'first\n' > d1/x.txt; printf 'second\n' > d2/x.txt
structured-zstd -f --rm --output-dir-flat out d1/x.txt d2/x.txt   # exit 0
# d1, d2 empty; out/x.txt.zst decodes to "second"

Upstream reports the same for the reference command: meta/zstd#4854 (fix proposed in meta/zstd#4855).

Fix

  • Before anything is written or removed, derive every input's output path and compare the names in the flat directory. With --rm, a collision is an error and the run stops with both sources in place. Without --rm, the warning and last-output-wins behaviour stay.
  • While the run writes, remember the identity (device and inode, or the Windows file ID) of every output it creates. With --rm, an input whose output is a file this run already wrote is refused and kept. This covers the names a byte comparison cannot judge (case or Unicode normalisation ignored by the file system) without guessing per platform, so a case-sensitive volume is never refused for names that are really distinct.

Acceptance

  • -f --rm --output-dir-flat out d1/x.txt d2/x.txt exits non-zero, writes nothing and keeps both sources.
  • A case-only difference on a case-insensitive file system exits non-zero, keeps the first archive and the second source.
  • Without --rm the run still writes the last output and warns, as the reference command does.
  • Regression tests on temporary copies cover compression and decompression.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions