Skip to content

cli: ignore trailing data after the last frame, and report corrupted archives - #4842

Open
Karaammm wants to merge 2 commits into
meta:devfrom
Karaammm:trailing-data-warning
Open

Karaammm wants to merge 2 commits into
meta:devfrom
Karaammm:trailing-data-warning

Conversation

@Karaammm

Copy link
Copy Markdown

What and why

When decompressing an archive containing valid zstd frames followed by arbitrary data, the CLI previously reported the input as an unsupported format or unknown header, even though all valid frames had been decompressed successfully.

This change improves the CLI behavior in two cases:

  • Trailing data after at least one successfully decoded frame is treated as trailing data rather than an unsupported format.
  • The CLI emits a warning:
    zstd: WARNING: <file>: trailing data after the last frame is ignored
  • Decompression succeeds with exit status 0 when only trailing data is encountered.
  • The source file is preserved when --rm is used and trailing data was ignored.
  • Truncated or corrupted zstd frames report how many bytes were decoded before the error:
    zstd: <file>: corrupted or incomplete archive; decoded <N> bytes
  • Corrupted or incomplete frames continue to return a failure status.
  • Inputs that never contained a valid frame retain their existing unsupported format and unknown header errors.
  • Pass-through mode continues to copy the input, including trailing data.

The implementation tracks whether a frame has already been decoded and records when trailing data was ignored. It also classifies the decoder errors that indicate corruption or an incomplete frame and reports the amount of output decoded before the failure.

The change includes CLI regression tests covering:

  • Unsupported input with no valid frame
  • Short trailing data
  • Longer trailing data
  • Multiple frames followed by trailing data
  • Input from stdin
  • Test mode
  • Quiet mode
  • Pass-through mode
  • --rm behavior for trailing data
  • Truncated frames
  • Corrupted frames
  • Decoded-byte reporting

Validation performed:

make -C programs zstd
./tests/cli-tests/run.py \
  --zstd ./programs/zstd \
  decompression/trailing-garbage.sh \
  decompression/corrupted-archive.sh

Result:

PASS: decompression/corrupted-archive.sh
PASS: decompression/trailing-garbage.sh
PASSED all 2 tests!

One limitation is that arbitrary trailing data and a damaged header of a later frame are not always distinguishable once the later frame's magic number has been corrupted. The current behavior follows the requested trailing-data behavior: the already decoded output is kept, a warning is emitted, and the source is preserved when --rm is used.

Checklist

[x] I searched open pull requests and issues: this problem is not already being addressed.
[x] Bug fix: it comes with a reproducer or a test that fails without the fix.
[ ]Performance change: numbers from repeated runs are included, with CPU, compiler, flags and data.
A gain in one configuration is not a general gain: see the Performance section of CONTRIBUTING.md.
[x] I stated below whether AI or other automated tools were used.

Tools used

Claude Code (Claude Sonnet 5.5) was used to find the code path, write the documentation, and the
tests. I ran and verified the tests locally.

Related to #4831

@meta-cla meta-cla Bot added the CLA Signed label Oct 14, 2026
@tansy

tansy commented Oct 14, 2026 •

Copy link
Copy Markdown

I thought I had to make additional changes. I'll make a new PR right now

yes, but as explained here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants