Add automatic download and evaluation of pretrained policies

[acrlw]

Description

Add eval-policy --pretrained <model-id> to download a policy's weights and matching configurations, resolve required robot assets, and run the existing native evaluator from one command.

embodichain eval-policy --pretrained g1-flat-ppo-newton \
  --viewer --command 0.3 0 0 --device cuda:0 --sim-device gpu

• Download only the selected policy and reuse cached files on subsequent runs. Use a fixed download endpoint so incompatible mirror settings do not break evaluation.
• Use a tested model version by default, with --revision to select another version.
• Support Viewer and Headless evaluation. Locomotion model bundles default to fast-rt.
• Keep evaluation reports separate from model caches and record the model ID and version. --cache-dir and --output control their locations.

The CLI guide, public API documentation, tests and download dependency declaration are updated accordingly.

Validation:

• With a mirror override configured, both default and named versions downloaded successfully into empty caches and matched the published bundle.
• All 12 policies completed 1,000 Viewer control steps each with FastRT and no task failures, using DexSim 0.5.0, Warp 1.15.0 and Newton 1.4.0.
• A G1 Headless batch started with empty model and asset caches and completed one 1,000-step episode on each backend.
• All 12 downloaded bundles matched the prepared files; complete model caches resolved with network requests disabled.
• 47 targeted asset and policy-evaluation tests, 40 agent-context tests and 8 API-documentation tests passed.
• Black, diff checks and public API documentation coverage (2258/2258) passed. Sphinx built successfully with warnings; none referenced the added API or changed CLI guide.

Type of change

• New feature (non-breaking change which adds functionality)

Screenshots

Not applicable: this extends the existing evaluation command.

Checklist

• I have run the black . command to format the code base.
• I reviewed affected documentation and agent context, updated it where needed, or explained why no update was needed.
• Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py), if applicable
• I have added tests that prove my fix is effective or that my feature works
• Dependencies have been updated, if applicable.

[greptile-apps]

Confidence Score: 4/5

The PR does not appear safe to merge until pretrained downloads handle valid manifest-referenced files.

Findings

1.  Valid manifest files go missing ▶
2.  Cache reuse test misses requests ▶

Fix with agent prompt

### Issue 1
embodichain/data/assets/policy_assets.py:35-41
If a bundle uses paths produced by `write_run_manifest()`, such as `configs/gym.yaml` or `checkpoints/policy.pt`, this fixed list does not download them. `RunManifest.load()` requires every referenced file to exist, so `eval-policy --pretrained` fails with `FileNotFoundError` after downloading the listed files. Download the files named by the manifest instead.

### Issue 2
tests/data/assets/test_policy_assets.py:78-81
The fake downloader leaves existing files untouched, and this test checks only the path and checkpoint modification time. A repeat run could still make network requests without failing the test, leaving the promised cache reuse unverified. Add a check of repeat-run request behavior at the Hub boundary.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR adds a command to download a versioned pretrained policy bundle and evaluate it with the existing native runtime.

• It adds model-cache and report locations, provenance recording, CLI documentation, and focused tests.
• The changes since the previous review pin Hub requests to the canonical endpoint.

_{Reviews (2) · Last reviewed commit: "fix(rl): use a stable endpoint for pretr..."}

[greptile-apps]

Valid manifest files go missing If a bundle uses paths produced by write_run_manifest(), such as configs/gym.yaml or checkpoints/policy.pt, this fixed list does not download them. RunManifest.load() requires every referenced file to exist, so eval-policy --pretrained fails with FileNotFoundError after downloading the listed files. Download the files named by the manifest instead.

Prompt To Fix With AI

This is a comment left during a code review.
Path: embodichain/data/assets/policy_assets.py
Line: 33-39

Comment:
**Valid manifest files go missing** If a bundle uses paths produced by `write_run_manifest()`, such as `configs/gym.yaml` or `checkpoints/policy.pt`, this fixed list does not download them. `RunManifest.load()` requires every referenced file to exist, so `eval-policy --pretrained` fails with `FileNotFoundError` after downloading the listed files. Download the files named by the manifest instead.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Cache reuse test misses requests The fake downloader leaves existing files untouched, and this test checks only the path and checkpoint modification time. A repeat run could still make network requests without failing the test, leaving the promised cache reuse unverified. Add a check of repeat-run request behavior at the Hub boundary.

Prompt To Fix With AI

This is a comment left during a code review.
Path: tests/data/assets/test_policy_assets.py
Line: 77-80

Comment:
**Cache reuse test misses requests** The fake downloader leaves existing files untouched, and this test checks only the path and checkpoint modification time. A repeat run could still make network requests without failing the test, leaving the promised cache reuse unverified. Add a check of repeat-run request behavior at the Hub boundary.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[yuecideng]

[P2] Use the canonical Hugging Face endpoint for policy downloads

When HF_ENDPOINT=https://hf-mirror.com, these calls inherit that mirror. For the pinned index.json, the mirror returns a 308 response without the Hub's commit/ETag metadata (get_hf_file_metadata reports both as None), so hf_hub_download fails before the documented eval-policy --pretrained command can start. The same URL on https://huggingface.co returns the pinned commit and ETag. solver_assets.py already pins the canonical endpoint for this reason. Please use that endpoint consistently for HfApi and both hf_hub_download calls, or otherwise handle incompatible mirrors.

[acrlw]

Fixed in 2c0fd74. HfApi and both hf_hub_download call sites now explicitly use https://huggingface.co, matching solver_assets.py. Standard HTTP proxy settings still apply.

Added regression coverage for pinned and named revisions with HF_ENDPOINT=https://hf-mirror.com. Both cases failed before the fix and pass now. I also verified real downloads into separate empty caches for the default revision and --revision main under that setting; all five bundle files matched the published model. All 47 targeted tests pass.