Skip to content

Fix MLU device utilities and test backend support - #14791

Open
SeptPonts wants to merge 2 commits into
huggingface:mainfrom
SeptPonts:feature/h3-mlu-support
Open

SeptPonts wants to merge 2 commits into
huggingface:mainfrom
SeptPonts:feature/h3-mlu-support

Conversation

@SeptPonts

@SeptPonts SeptPonts commented Sep 16, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes #14779.

Maintainer acknowledgment:
#14779 (comment)

Complete the existing MLU backend support by:

  • Registering MLU operations in the production and testing backend utility tables.
  • Releasing unused MLU allocator cache in training_utils.free_memory().
  • Selecting an available MLU automatically in the test utilities.

On MLU590-M9DK, the six production helpers that previously raised TypeError now succeed. After deleting a 64 MiB tensor, free_memory() releases the reserved allocator memory. The existing H3 training test passes on MLU with automatic device selection.

Before/after measurements and environment details are recorded in #14779.

Validation

Passed locally with the repository-pinned Ruff 0.9.10:

  • make style and make fix-copies
  • make quality
  • python utils/check_copies.py
  • python utils/check_dummies.py
  • python utils/check_support_list.py
  • python utils/check_forward_call_docstrings.py
  • make deps_table_check_updated
  • git diff --check

make style also proposed an unrelated, pre-existing LTX2 import-table reorder. It was excluded from this PR; make quality passed on the final three-file change.

Before/after hardware validation used MLU590-M9DK, Torch 2.12.1 and torch_mlu 1.34.1:

  • Six MLU backend utility calls that raised TypeError on the parent now succeed.

  • free_memory() releases the allocator cache after a deleted 64 MiB allocation: reserved memory changes from 67,108,864 bytes on the parent to zero with the patch.

  • The existing H3 training test passes with automatic MLU selection:

    python -B -m pytest --assert=plain -p no:cacheprovider -q \
      tests/models/transformers/test_models_transformer_minimax_h3.py::TestMiniMaxH3TransformerTraining::test_training

The full test suite and CUDA/NPU runtime suites were not run. Hardware validation predates the final import-name sorting correction; no runtime logic changed afterward.

Self-review

Applied .ai/skills/self-review/SKILL.md to the entire three-file diff. No blocking correctness issues or unused additions were found. The known import-order lint failure has been corrected and quality checks pass.

One initialization assumption remains for maintainer review: MLU table entries expect torch_mlu registration before the utility modules are imported. Automatic extension loading supplies this in the tested environment; late extension loading with autoload disabled is not covered.

This fixes device utilities and test-backend selection. No public signatures, dtype policies or determinism settings change.

Before submitting

  • Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
    • Did you read the Coding with AI agents guide?
    • Did you run the self-review skill on the diff?
    • Did you share the final self-review notes in the PR description or a comment?
  • Did you read the contributor guideline?
  • Did you read our philosophy doc? (important for complex PRs)
  • Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?
  • Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sayakpaul Following up on the discussion in #14779.

@sayakpaul sayakpaul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread tests/testing_utils.py
# Function definitions
BACKEND_EMPTY_CACHE = {
"cuda": torch.cuda.empty_cache,
"mlu": getattr(getattr(torch, "mlu", None), "empty_cache", None),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this the case? is empty_cache not available generally in Torch MLU?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, empty_cache() is available in torch_mlu, and I verified it on hardware.

The guard is for the optional backend namespace: unlike the built-in torch.cuda and torch.xpu modules, torch.mlu is registered by the external torch_mlu package through PyTorch鈥檚 PrivateUse1 integration. Without the extension loaded, accessing torch.mlu.empty_cache while constructing this module-level dictionary raises AttributeError: module 'torch' has no attribute 'mlu'. This prevents the shared utilities from being imported even for CPU/CUDA tests, before any MLU operation is requested.

I followed the guarded-access pattern already used for Neuron鈥檚 device_count and synchronize entries in src/diffusers/utils/torch_utils.py. The same reasoning applies to the device_count comment below.

Comment thread tests/testing_utils.py
}
BACKEND_DEVICE_COUNT = {
"cuda": torch.cuda.device_count,
"mlu": lambda: getattr(getattr(torch, "mlu", None), "device_count", lambda: 0)(),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@sayakpaul
sayakpaul requested a review from DN6 September 17, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MLU backend utilities fail despite MLU device detection

2 participants