fix: prevent incomplete Search-R1 data cache files

[Bluu (Bluuok)]

Failed Search-R1 downloads or index assembly can leave non-empty partial cache files that subsequent runs treat as valid. Wildcard shard assembly also includes unrelated part_* files.

Write downloads and the assembled index into same-directory temporary files, clean up failures, and publish each final path only after success. Assemble the index from the two expected shards (part_aa, part_ab). Existing non-empty cache files remain reusable.

Validation in the frozen Linux CPU environment:

python -m pytest tests/examples/test_search_r1_data.py -q -p no:cacheprovider
python -m ruff check tests/examples/test_search_r1_data.py
python -m ruff format --check tests/examples/test_search_r1_data.py
python -m pre_commit run --files examples/search_r1/data_process.sh tests/examples/test_search_r1_data.py
git diff --check

All 4 tests passed. The original download/stale-shard regressions failed on unchanged main. The partial-assembly regression also failed on the previous branch version (1 failed / 3 passed): it emits the first shard and then returns a read error, verifies that no final or temporary index remains, and verifies that a subsequent run recovers the complete index. Scoped Pyright and syntax checks on repository-LF content passed.

Tests execute the full Bash script with isolated Conda/downloader commands and tiny gzip fixtures. No actual dataset download, external service or GPU training was run. Fork CI awaits maintainer approval. AI assistance was used; both the final script and regression tests were reviewed.

[Copilot]

Copilot review overview

🟡 Changes recommended

Index assembly can still leave a non-empty partial cache file when concatenation fails.

Review effort: Balanced
Findings: 1

Open (1)

• Avoid caching partial index when shard assembly fails · New

What changed in this PR

Improves Search-R1 cache reliability by publishing successful downloads atomically and restricting index assembly to expected shards.

Changes:

• Downloads through temporary files with failure cleanup.
• Uses only part_aa and part_ab for index assembly.
• Adds end-to-end regression tests for curl/wget failures and stale shards.

File	Description
examples/​search_r1/​data_process.sh	Adds atomic downloads and explicit shard assembly.
tests/​examples/​test_search_r1_data.py	Tests retries, cache preservation, and index contents.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.