Pass maxdepth through in AsyncFileSystem._rm - #2106
Conversation
AsyncFileSystem._rm accepted maxdepth into **kwargs and never used it, so async filesystems deleted deeper than the caller asked for. The sync twin passes it to expand_path; this does the same. Adds test_rm_honours_maxdepth.
fallenmi
left a comment
There was a problem hiding this comment.
Verified on exact head d8402ab64320792f22f4c3b1e0444674bb3d0fff against merge base c45f6011c37a74e4cf64d4378bad3e0af38382d0.
Using the same three-level AsyncFileSystem primitive as the regression, the base schedules all six paths for rm("/root", recursive=True, maxdepth=1), including /root/d1/b.txt and /root/d1/d2/c.txt. This head schedules only /root, /root/a.txt, and /root/d1; with maxdepth=None, it still schedules all six, so the unbounded path is unchanged. The new regression passes, and the full fsspec/tests/test_async.py target is 28 passed on Python 3.13.5.
The argument is now consumed by _expand_path instead of leaking through per-file kwargs, matching synchronous AbstractFileSystem.rm. Looks correct to me.
Reviewed with OpenAI Codex assistance; I independently inspected the source and ran the checks above.
AsyncFileSystem._rmacceptsmaxdepthinto**kwargsand never uses it, so async filesystems delete deeper than the caller asked for.The sync twin passes it on:
maxdepthbounds which paths get collected for deletion, and its own docstring says that without it "there will be no limit and infinite recursion may be possible". On the async side it lands in**kwargsand is forwarded to_rm_file, which has no use for it, while_expand_pathruns unbounded.With a tree of
/root/a.txt,/root/d1/b.txtand/root/d1/d2/c.txt, callingrm("/root", recursive=True, maxdepth=1)on the same primitives:Three files the bound was meant to protect. That's data loss. This reaches any backend implementing
_rm_fileand inheriting_rm, which is the usual pattern, anddelete()is an alias ofrmcarryingmaxdepthtoo.Test is
test_rm_honours_maxdepth, using the same shape as the existing_CatRangesFScase. Suite goes 1556 to 1557 onfsspec/testsandfsspec/implementations/tests. Reverting onlyasyn.pyfails that one test, so it's the fix being measured.ruff checkandruff formatare clean under the 0.14.3 you pin in pre-commit.This only touches the base class. A backend overriding
_rmwith its own signature needs the same change, and I have not run this against a real cloud backend.