Skip to content

Add Arm Neoverse V3/V3AE recognition - #435

Open
ashuie wants to merge 4 commits into
pytorch:mainfrom
ashuie:add-neoverse-v3-support
Open

ashuie wants to merge 4 commits into
pytorch:mainfrom
ashuie:add-neoverse-v3-support

Conversation

@ashuie

@ashuie ashuie commented Sep 21, 2026 •

Copy link
Copy Markdown

Support the Arm Neoverse V3 and V3AE architecture.

Verified on a Neoverse V3 gv5 host:

  • MIDR: 0x410fd841 (r0p1)
  • L1 data: 64 KiB, 4-way
  • L1 instruction: 64 KiB, 4-way
  • private L2: 2048 KiB, 8-way per core

For reference: Adding Neoverse V2 support: #194

@meta-cla

meta-cla Bot commented Sep 21, 2026

Copy link
Copy Markdown

Hi @ashuie!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@meta-cla meta-cla Bot added the cla signed label Sep 21, 2026
@meta-cla

meta-cla Bot commented Sep 21, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@ashuie
ashuie marked this pull request as ready for review September 21, 2026 20:59
@Rohanjames1997

Copy link
Copy Markdown
Contributor

@malfet @zhili03 @fbarchard - could you take a look at this PR? Thanks!

Comment thread src/arm/linux/aarch64-isa.c
Comment thread src/arm/linux/aarch64-isa.c

@nSircombe nSircombe left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will pick up V3 but miss V3AE. Could you add the MIDR for v3ae? (0x410FD830, I think - Implementor = 0x41, part = 0xD83).

see OpenMathLib/OpenBLAS#5759

@ashuie

ashuie commented Sep 24, 2026

Copy link
Copy Markdown
Author

@nSircombe Thanks, I've added V3AE.

@ashuie
ashuie requested a review from nSircombe September 24, 2026 16:12
Comment thread src/arm/cache.c

@Radu2k Radu2k left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you! 🙂

@Radu2k

Radu2k commented Sep 25, 2026

Copy link
Copy Markdown

Might want to update the PR body as well.

@ashuie ashuie changed the title Add Arm Neoverse V3 recognition Add Arm Neoverse V3/V3AE recognition Sep 25, 2026
@aditew01

Copy link
Copy Markdown

Cc: @malfet @huydhn can you please help with the review

pytorchmergebot pushed a commit to pytorch/pytorch that referenced this pull request Sep 25, 2026
Requires `is_arm_neoverse()` to recognize Neoverse V3: pytorch/cpuinfo#435

On Arm Neoverse systems built with the oneDNN ACL backend, decline BF16 CPUBlas GEMMs when `n == 1` and `alpha == 1` so the existing native GEMV fallback is used. `n == 1` makes the operation a matrix-vector product, and these shapes are common during single-token transformer decode.
In official Linux AArch64 wheels, the fallback is OpenBLAS `sbgemm_` when available. Otherwise, CPUBlas uses ATen `gemm_stub`.
`is_arm_neoverse()` is required because this fallback only benefits Arm.

Tested on Neoverse V3 - c8g.16xlarge and Neoverse V2 - c9g.16xlarge.
GCC 13.3 and oneDNN 3.12.0

Workload: GPT-2 BF16 single token decode on one pinned thread

| CPU | Before | After | Speedup |
|---|---:|---:|---:|
| Graviton 5 | 56.0842 ms | 10.5462 ms | **5.32x** |
| Graviton 4 | 75.9550 ms | 14.1522 ms | **5.37x** |

Across other thread counts:
| CPU | Threads | Before | After | Speedup |
|---|---:|---:|---:|---:|
| Graviton 5 | 8 | 11.2110 ms | 4.0912 ms | **2.74x** |
| Graviton 5 | 16 | 8.2988 ms | 4.5431 ms | **1.83x** |
| Graviton 5 | 32 | 7.0627 ms | 4.2545 ms | **1.66x** |
| Graviton 5 | 64 | 6.1616 ms | 4.2753 ms | **1.44x** |

Correctness:
CPUBlas results were compared against the existing oneDNN results across every transpose layout, `beta=0` and `beta=1`, and both aligned and non-aligned dimensions.

Pull Request resolved: #198411
Approved by: https://github.com/aditew01, https://github.com/Skylion007
Comment thread src/arm/cache.c

@fbarchard fbarchard left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Consider adding a unit test at the end of test/arm-cache.cc (line 2152) to exercise cpuinfo_arm_decode_cache for
cpuinfo_uarch_neoverse_v3 (0xD84) and V3AE (0xD83):

```cpp
TEST(ARM, neoverse_v3) {
    const struct cpuinfo_arm_chipset chipset = {
        .vendor = cpuinfo_arm_chipset_vendor_unknown,
        .series = cpuinfo_arm_chipset_series_unknown,
    };

    struct cpuinfo_cache l1i = {0};
    struct cpuinfo_cache l1d = {0};
    struct cpuinfo_cache l2 = {0};
    struct cpuinfo_cache l3 = {0};
    cpuinfo_arm_decode_cache(
        cpuinfo_uarch_neoverse_v3, 4, UINT32_C(0x410FD841), &chipset, 0, 4, &l1i, &l1d, &l2, &l3);

    EXPECT_EQ(64 * 1024, l1i.size);
    EXPECT_EQ(4, l1i.associativity);
    EXPECT_EQ(64, l1i.line_size);
    EXPECT_EQ(64 * 1024, l1d.size);
    EXPECT_EQ(4, l1d.associativity);
    EXPECT_EQ(64, l1d.line_size);
    EXPECT_EQ(1024 * 1024, l2.size);
    EXPECT_EQ(8, l2.associativity);
    EXPECT_EQ(64, l2.line_size);
    EXPECT_EQ(CPUINFO_CACHE_INCLUSIVE, l2.flags);
    EXPECT_EQ(0, l3.size);
}
```

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants