Add SIMD algebraic intrinsics by calebzulawski · Pull Request #164952 · rust-lang/rust

calebzulawski · 2026-10-26T03:15:39Z

Added algebraic intrinsics for SIMD. First half of rust-lang/portable-simd#539

rustbot · 2026-10-26T03:15:43Z

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

Some changes occurred to the CTFE machinery

Some changes occurred to the platform-builtins intrinsics. Make sure the
LLVM backend as well as portable-simd gets adapted for the changes.

cc @antoyo, @GuillaumeGomez, @bjorn3, @programmerjake

Some changes occurred to the intrinsics. Make sure the CTFE / Miri interpreter
gets adapted for the changes, if necessary.

cc @rust-lang/miri, @RalfJung, @oli-obk, @lcnr

rustbot · 2026-10-26T03:15:46Z

r? @davidtwco

rustbot has assigned @davidtwco.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: compiler
compiler expanded to 77 candidates
Random selection from 21 candidates

programmerjake · 2026-10-26T03:59:50Z

+                        sym::simd_reduce_add_algebraic | sym::simd_reduce_mul_algebraic
+                    );
                let mut res = init;
                for i in 0..op_len {


the unordered ops should probably also pick a random order to reduce in, but imo that doesn't need to be part of this PR.

View changes since the review

In const-eval the order must be deterministic so we can't randomize.

However, Miri needs to randomize them. With this PR Miri will start running them, without proper randomization. That's a bug introduced by this PR. So I disagree, if we have Miri support here then it must be correct.

RalfJung · 2026-10-26T07:46:14Z

+                    sym::simd_add_algebraic => Op::MirOp(BinOp::Add),
+                    sym::simd_sub_algebraic => Op::MirOp(BinOp::Sub),
+                    sym::simd_mul_algebraic => Op::MirOp(BinOp::Mul),
+                    sym::simd_div_algebraic => Op::MirOp(BinOp::Div),
+                    sym::simd_rem_algebraic => Op::MirOp(BinOp::Rem),


Seems cleaner to represent this as a MirOpFloatAlgebraic or so to avoid having to match twice.

View changes since the review

RalfJung · 2026-10-26T07:52:39Z

+                            if val.layout.ty.is_floating_point() && nondet {
+                                val = M::apply_float_nondet(self, val)?;
+                            }


To ensure that this really makes things non-deterministic, please add a test like this in the tests/pass/intrinsics/portable-simd.rs file.

View changes since the review

rustbot · 2026-10-26T23:53:28Z

miri is developed in its own repository. If the Miri part of this change can be broken out, consider making this change to rust-lang/miri instead. However, if Miri needs adjusting for rustc changes, just ignore this message.

cc @rust-lang/miri

rustbot · 2026-10-27T00:32:10Z

rustc_codegen_gcc is developed in its own repository. If possible, consider making this change to rust-lang/rustc_codegen_gcc instead.

cc @antoyo, @GuillaumeGomez

rustc_codegen_cranelift is developed in its own repository. If possible, consider making this change to rust-lang/rustc_codegen_cranelift instead.

cc @bjorn3

programmerjake · 2026-10-27T02:36:09Z

+                };
+
+                let mut res = init;
+                for i in M::nondet_index_iterator(self, op_len) {


maybe add a comment noting that this doesn't fully randomize the reduce order, since it only randomizes the order of the inputs then does an ordered reduce -- that will never give you an evaluation order like (a + b) + (c + d).

View changes since the review

an example implementation that does cover all possible evaluation orders: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=c2f778ccd6b16d881b5afac8200ffaf7

pub fn random_reduce<T>(mut inputs: Vec<T>, mut reduce: impl FnMut(T, T) -> T) -> Option<T> { while inputs.len() >= 2 { let lhs_index = rand::random_range(0..inputs.len()); let lhs = inputs.swap_remove(lhs_index); let rhs_index = rand::random_range(0..inputs.len()); let rhs = inputs.swap_remove(rhs_index); inputs.push(reduce(lhs, rhs)); } inputs.pop() }

I'm curious if this is actually necessary, miri's float nondeterminism doesn't generate every possible distribution of nondeterminism

maybe not, but in the case of catastrophic cancellation, you can get wildly different results for the different evaluation orders so imo it would be useful for miri to be able to give you those.

…educe order, and add non-determinism miri test for SIMD

… regular operations

Co-authored-by: Ralf Jung <post@ralfj.de>

rustbot · 2026-10-31T03:54:08Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

calebzulawski · 2026-10-31T03:55:58Z

I think #164924 introduced a minor break to cg_gcc vector frem, I repaired that here

RalfJung · 2026-10-31T08:41:21Z

+    #[inline(always)]
+    fn nondet_index_iterator(ecx: &mut InterpCx<'tcx, Self>, len: u64) -> impl Iterator<Item = u64> + use<'tcx> {
+        let mut indices = (0..len).collect::<Vec<u64>>();
+        if ecx.machine.float_nondet {


Since the implementation is float-specific, let's call this unordered_float_reduction_iter or something like that.

View changes since the review

RalfJung · 2026-10-31T08:42:09Z

+
+                let (op, op_len) = self.project_to_simd(&args[0])?;
+
+                let scalar = |f: Double| {


If you are creating the initial value via casts anyway, might as well use int_to_int_or_float and cast from an integer, that should be a bit simpler.

View changes since the review

RalfJung · 2026-10-31T08:42:53Z

+                for i in M::nondet_index_iterator(self, op_len) {
+                    let op = self.read_immediate(&self.project_index(&op, i)?)?;
+                    res = self.binary_op(mir_op, &res, &op)?;
+                    if dest.layout.ty.is_floating_point() {


This should only be called on float types anyway, right?

View changes since the review

RalfJung · 2026-10-31T08:46:19Z

+
+                let (op, op_len) = self.project_to_simd(&args[0])?;
+
+                let scalar = |f: Double| {


Suggested change

let scalar = |f: Double| {

// Compute starting value and reduction MIR op.

let from_int = |i: i32| {

View changes since the review

rustbot assigned davidtwco Oct 26, 2026

calebzulawski force-pushed the feature/algebraic-simd branch from e92596e to 9d8b178 Compare October 26, 2026 03:30

programmerjake reviewed Oct 26, 2026

View reviewed changes

RalfJung reviewed Oct 26, 2026

View reviewed changes

Comment thread compiler/rustc_const_eval/src/interpret/intrinsics/simd.rs Resolved

RalfJung reviewed Oct 26, 2026

View reviewed changes

Comment thread library/core/src/intrinsics/simd/mod.rs Resolved

bjorn3 reviewed Oct 26, 2026

View reviewed changes

Comment thread compiler/rustc_codegen_llvm/src/intrinsic.rs Resolved

calebzulawski force-pushed the feature/algebraic-simd branch from 3125bca to 6796d73 Compare October 27, 2026 00:36

programmerjake reviewed Oct 27, 2026

View reviewed changes

bjorn3 reviewed Oct 27, 2026

View reviewed changes

Comment thread compiler/rustc_codegen_cranelift/src/intrinsics/simd.rs Resolved

RalfJung reviewed Oct 29, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

calebzulawski force-pushed the feature/algebraic-simd branch from 4db7858 to 3067057 Compare October 31, 2026 02:23

This comment has been minimized.

Sign in to view

calebzulawski and others added 7 commits October 30, 2026 23:39

Add algebraic simd intrinsics

40d741c

Split ordered and unordered intrinsics, implement non-deterministic r…

b2b4efe

…educe order, and add non-determinism miri test for SIMD

Implement SIMD algebraic intrinsics in cg_clif and cg_gcc by emitting…

3ff298e

… regular operations

Update library/core/src/intrinsics/simd/mod.rs

0bf20d3

Co-authored-by: Ralf Jung <post@ralfj.de>

Disable unordered reduces when nondet is disabled in miri

537e8cd

Undo const unordered operations

b8ffa42

Make match arms a little neater

909538c

calebzulawski added 3 commits October 30, 2026 23:39

Don't use host floats

d0d112a

Test f16/f128, and check that results match the expected value

d5da8b1

Fix cg_gcc frem

95f5629

calebzulawski force-pushed the feature/algebraic-simd branch from 3067057 to 95f5629 Compare October 31, 2026 03:54

RalfJung reviewed Oct 31, 2026

View reviewed changes


		let (op, op_len) = self.project_to_simd(&args[0])?;

		let scalar = \|f: Double\| {

	let scalar = \|f: Double\| {
	// Compute starting value and reduction MIR op.
	let from_int = \|i: i32\| {

Conversation

calebzulawski commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Oct 26, 2026

Uh oh!

rustbot commented Oct 26, 2026

Uh oh!

programmerjake Oct 26, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 26, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 26, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rustbot commented Oct 26, 2026

Uh oh!

rustbot commented Oct 27, 2026

Uh oh!

programmerjake Oct 27, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

programmerjake Oct 27, 2026

Choose a reason for hiding this comment

Uh oh!

calebzulawski Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

programmerjake Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

rustbot commented Oct 31, 2026

Uh oh!

calebzulawski commented Oct 31, 2026

Uh oh!

RalfJung Oct 31, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 31, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 31, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 31, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

calebzulawski commented Oct 26, 2026 •

edited

Loading

programmerjake Oct 26, 2026 •

edited by rustbot

Loading

RalfJung Oct 26, 2026 •

edited

Loading

RalfJung Oct 26, 2026 •

edited by rustbot

Loading

RalfJung Oct 26, 2026 •

edited by rustbot

Loading

programmerjake Oct 27, 2026 •

edited by rustbot

Loading

RalfJung Oct 31, 2026 •

edited by rustbot

Loading

RalfJung Oct 31, 2026 •

edited by rustbot

Loading

RalfJung Oct 31, 2026 •

edited by rustbot

Loading

RalfJung Oct 31, 2026 •

edited by rustbot

Loading