Conversation
| # Function definitions | ||
| BACKEND_EMPTY_CACHE = { | ||
| "cuda": torch.cuda.empty_cache, | ||
| "mlu": getattr(getattr(torch, "mlu", None), "empty_cache", None), |
There was a problem hiding this comment.
Why is this the case? is empty_cache not available generally in Torch MLU?
There was a problem hiding this comment.
Yes, empty_cache() is available in torch_mlu, and I verified it on hardware.
The guard is for the optional backend namespace: unlike the built-in torch.cuda and torch.xpu modules, torch.mlu is registered by the external torch_mlu package through PyTorch鈥檚 PrivateUse1 integration. Without the extension loaded, accessing torch.mlu.empty_cache while constructing this module-level dictionary raises AttributeError: module 'torch' has no attribute 'mlu'. This prevents the shared utilities from being imported even for CPU/CUDA tests, before any MLU operation is requested.
I followed the guarded-access pattern already used for Neuron鈥檚 device_count and synchronize entries in src/diffusers/utils/torch_utils.py. The same reasoning applies to the device_count comment below.
| } | ||
| BACKEND_DEVICE_COUNT = { | ||
| "cuda": torch.cuda.device_count, | ||
| "mlu": lambda: getattr(getattr(torch, "mlu", None), "device_count", lambda: 0)(), |
What does this PR do?
Fixes #14779.
Maintainer acknowledgment:
#14779 (comment)
Complete the existing MLU backend support by:
training_utils.free_memory().On MLU590-M9DK, the six production helpers that previously raised
TypeErrornow succeed. After deleting a 64 MiB tensor,free_memory()releases the reserved allocator memory. The existing H3 training test passes on MLU with automatic device selection.Before/after measurements and environment details are recorded in #14779.
Validation
Passed locally with the repository-pinned Ruff 0.9.10:
make styleandmake fix-copiesmake qualitypython utils/check_copies.pypython utils/check_dummies.pypython utils/check_support_list.pypython utils/check_forward_call_docstrings.pymake deps_table_check_updatedgit diff --checkmake stylealso proposed an unrelated, pre-existing LTX2 import-table reorder. It was excluded from this PR;make qualitypassed on the final three-file change.Before/after hardware validation used MLU590-M9DK, Torch 2.12.1 and torch_mlu 1.34.1:
Six MLU backend utility calls that raised
TypeErroron the parent now succeed.free_memory()releases the allocator cache after a deleted 64 MiB allocation: reserved memory changes from 67,108,864 bytes on the parent to zero with the patch.The existing H3 training test passes with automatic MLU selection:
The full test suite and CUDA/NPU runtime suites were not run. Hardware validation predates the final import-name sorting correction; no runtime logic changed afterward.
Self-review
Applied
.ai/skills/self-review/SKILL.mdto the entire three-file diff. No blocking correctness issues or unused additions were found. The known import-order lint failure has been corrected and quality checks pass.One initialization assumption remains for maintainer review: MLU table entries expect
torch_mluregistration before the utility modules are imported. Automatic extension loading supplies this in the tested environment; late extension loading with autoload disabled is not covered.This fixes device utilities and test-backend selection. No public signatures, dtype policies or determinism settings change.
Before submitting
self-reviewskill on the diff?documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@sayakpaul Following up on the discussion in #14779.