offload: pass the linked device image to the host pass by sgasho · Pull Request #164896 · rust-lang/rust

sgasho · 2026-10-25T10:03:09Z

Closes #164057
Follow-up to: #164455

The host pass now takes the linked device output (<crate>.ptx for NVPTX, <crate>.elf for AMDGPU) instead of the device bitcode.

The device crate must now be built as a cdylib. rustc only passes --export-symbol to llvm-bitcode-linker for cdylibs, so a bin would lose its kernels to internalization.

I had to add [lib] in Cargo.toml

[lib]
name = "sample_offload"
path = "src/main.rs"
crate-type = ["cdylib"]

Without it, the runtime fails at startup

PluginInterface error: Failure to init kernel: "requested object was not found in the binary image" error in cuModuleGetFunction('_RNvC14sample_offload6kernel'): named symbol not found
omptarget error: Failed to load kernel _RNvC14sample_offload6kernel

I’m not sure whether bin device crates should export their kernels too, so I left the linker untouched for now.

The second commit fixes another runtime error without --emit=llvm-bc/llvm-ir. (my understanding is that we should not have to set these after this PR, because no device.bc anymore)
The “already wrapped” check in handle_offload relied on a value name, which LLVM drops unless --emit=llvm-bc/llvm-ir is given, so the kernel got wrapped twice.

I verified on UofT nvidia gpunode (amdgpunodes was not available for my account).

r? ZuseZ4

ZuseZ4 · 2026-10-25T20:31:02Z

Thank you for also solving the fixme along the way. I'm at the llvm dev till wednesday, but I'll try to test it a bit in between.

@kjetilkjeka you had some ideas on how to configure cpu vs gpu builds, right?

fyi @xkevio

kjetilkjeka · 2026-10-26T18:53:17Z

For my cargo experiment with offload support this is a step in the right direction, with a little regression.

It fits well within the model of building a single "cdylib" (ptx/elf) for the device compilation per binary (as opposed to one per crate).

The unfortunate regression is of course that a crate with a single main.rs binary requires the mentioned workaround. I believe the solution is to tell rustc that when -Zoffload=Device is set then what would have been executables on the host side should be treated as cdylib on the device side. This is true for both normal executables and when building tests, for the root host unit we basically want a corresponding cdylib on the device side.

I checked out your PR and added a mapping in collect_crate_types from CrateType::Executable to CrateType::Cdylib and that made my basic examples build and run as expected using nvptx64 and an Nvidia GPU.

I haven't got far enough to test cargo test and don't know if that is within scope for offload right now. If it is there might be something like this that needs to be done for the test harness injection for device as well. We basically want to build a device cdylib built with --cfg test and that kind of things, but obviously without injecting the test harness for the offload device.

ZuseZ4 · 2026-10-27T20:28:33Z

Thanks. I agree that independently of the final design we almost never want to compile the GPU code as a normal binary, even if libc-for-gpu might allow us to do such cursed/funny things. So for now mapping the GPU cratetype seems like the right call, @sgasho can you try to add that? I'd expect that if we remap it early enough it should be unproblematic

sgasho · 2026-10-29T15:45:41Z

Thanks both. I added the Executable → Cdylib mapping in collect_crate_types,
as kjetilkjeka suggested, since that is where crate types are finalized.
I verified that sample offload works on an nvidia gpunode.

ZuseZ4 · 2026-10-29T17:24:11Z

Thanks, another hundred lines saved. I tested it on an AMD igpu, where it also works.
I left one nit, can you fold that into the commit which introduced the helper? Otherwise lgtm.

ZuseZ4 · 2026-11-02T12:38:58Z

@bors delegate r=me when rebased

rust-bors · 2026-11-02T12:39:01Z

Invalid command: Key-value argument r is not supported. Run @bors help or go to https://bors.rust-lang.org/help to see available commands.

ZuseZ4 · 2026-11-02T12:41:09Z

@bors delegate lgtm, after rebase

rust-bors · 2026-11-02T12:41:12Z

Unknown argument "lgtm,". Did you mean to use @bors tryorreview``? Run @bors help or go to https://bors.rust-lang.org/help to see available commands.

ZuseZ4 · 2026-11-02T12:44:15Z

I thought bors now allows comments following the command? But I guess that's just for r+. lgtm, after rebase

@bors delegate

rust-bors · 2026-11-02T12:44:17Z

✌️ @sgasho, you can now approve this pull request!

If @ZuseZ4 told you to "r=me" after making some further change, then please make that change and post @bors r=ZuseZ4.

View changes since this delegation.

sgasho · 2026-11-03T07:20:29Z

@bors r=ZuseZ4

rust-bors · 2026-11-03T07:20:33Z

📌 Commit c6749d8 has been tentatively approved by ZuseZ4

It will be put into the queue for this repository once PR CI succeeds.

offload: pass the linked device image to the host pass Closes rust-lang#164057 Follow-up to: rust-lang#164455 The host pass now takes the linked device output (`<crate>.ptx` for NVPTX, `<crate>.elf` for AMDGPU) instead of the device bitcode. The device crate must now be built as a cdylib. rustc only passes --export-symbol to llvm-bitcode-linker for cdylibs, so a bin would lose its kernels to internalization. I had to add [lib] in Cargo.toml ```toml [lib] name = "sample_offload" path = "src/main.rs" crate-type = ["cdylib"] ``` Without it, the runtime fails at startup ``` PluginInterface error: Failure to init kernel: "requested object was not found in the binary image" error in cuModuleGetFunction('_RNvC14sample_offload6kernel'): named symbol not found omptarget error: Failed to load kernel _RNvC14sample_offload6kernel ``` I’m not sure whether bin device crates should export their kernels too, so I left the linker untouched for now. The second commit fixes another runtime error without `--emit=llvm-bc`/`llvm-ir`. (my understanding is that we should not have to set these after this PR, because no device.bc anymore) The “already wrapped” check in handle_offload relied on a value name, which LLVM drops unless --emit=llvm-bc/llvm-ir is given, so the kernel got wrapped twice. I verified on UofT nvidia gpunode (amdgpunodes was not available for my account). r? ZuseZ4

…uwer Rollup of 15 pull requests Successful merges: - #165378 (rustfmt subtree update (v1.12.0)) - #149245 (Make inline attributes apply to the generated `poll` in async fn) - #164304 ([perf] Reuse CanonicalVarKinds if no new variables were added) - #165109 (Avoid building a region map when erasing bound regions) - #163393 (std: Use getexecpath() for current_exe() on OpenBSD) - #164896 (offload: pass the linked device image to the host pass) - #165358 (Reenable some cg_gcc tests that were disabled) - #165371 (Don't use `llvm_type` as the type of `static`s.) - #162232 (Replace `builtin #` with `k#`) - #162608 (Avoid oscillating line wrapping due to symbol indices) - #165029 (Use `MalformedLintAttributeInput` in more places) - #165203 (AIX: Implement TLS destructor guard in std and update dynamic-loading-cdylib test) - #165230 (Fix rigidity assertion ICE) - #165354 (Remove `AbiAlign`) - #165374 (Refactorings in preparation for the move trait)

offload: pass the linked device image to the host pass Closes rust-lang#164057 Follow-up to: rust-lang#164455 The host pass now takes the linked device output (`<crate>.ptx` for NVPTX, `<crate>.elf` for AMDGPU) instead of the device bitcode. The device crate must now be built as a cdylib. rustc only passes --export-symbol to llvm-bitcode-linker for cdylibs, so a bin would lose its kernels to internalization. I had to add [lib] in Cargo.toml ```toml [lib] name = "sample_offload" path = "src/main.rs" crate-type = ["cdylib"] ``` Without it, the runtime fails at startup ``` PluginInterface error: Failure to init kernel: "requested object was not found in the binary image" error in cuModuleGetFunction('_RNvC14sample_offload6kernel'): named symbol not found omptarget error: Failed to load kernel _RNvC14sample_offload6kernel ``` I’m not sure whether bin device crates should export their kernels too, so I left the linker untouched for now. The second commit fixes another runtime error without `--emit=llvm-bc`/`llvm-ir`. (my understanding is that we should not have to set these after this PR, because no device.bc anymore) The “already wrapped” check in handle_offload relied on a value name, which LLVM drops unless --emit=llvm-bc/llvm-ir is given, so the kernel got wrapped twice. I verified on UofT nvidia gpunode (amdgpunodes was not available for my account). r? ZuseZ4

…uwer Rollup of 14 pull requests Successful merges: - #165378 (rustfmt subtree update (v1.12.0)) - #149245 (Make inline attributes apply to the generated `poll` in async fn) - #164304 ([perf] Reuse CanonicalVarKinds if no new variables were added) - #165109 (Avoid building a region map when erasing bound regions) - #163393 (std: Use getexecpath() for current_exe() on OpenBSD) - #164896 (offload: pass the linked device image to the host pass) - #165358 (Reenable some cg_gcc tests that were disabled) - #165371 (Don't use `llvm_type` as the type of `static`s.) - #162232 (Replace `builtin #` with `k#`) - #162608 (Avoid oscillating line wrapping due to symbol indices) - #165029 (Use `MalformedLintAttributeInput` in more places) - #165203 (AIX: Implement TLS destructor guard in std and update dynamic-loading-cdylib test) - #165230 (Fix rigidity assertion ICE) - #165374 (Refactorings in preparation for the move trait)

offload: pass the linked device image to the host pass Closes rust-lang#164057 Follow-up to: rust-lang#164455 The host pass now takes the linked device output (`<crate>.ptx` for NVPTX, `<crate>.elf` for AMDGPU) instead of the device bitcode. The device crate must now be built as a cdylib. rustc only passes --export-symbol to llvm-bitcode-linker for cdylibs, so a bin would lose its kernels to internalization. I had to add [lib] in Cargo.toml ```toml [lib] name = "sample_offload" path = "src/main.rs" crate-type = ["cdylib"] ``` Without it, the runtime fails at startup ``` PluginInterface error: Failure to init kernel: "requested object was not found in the binary image" error in cuModuleGetFunction('_RNvC14sample_offload6kernel'): named symbol not found omptarget error: Failed to load kernel _RNvC14sample_offload6kernel ``` I’m not sure whether bin device crates should export their kernels too, so I left the linker untouched for now. The second commit fixes another runtime error without `--emit=llvm-bc`/`llvm-ir`. (my understanding is that we should not have to set these after this PR, because no device.bc anymore) The “already wrapped” check in handle_offload relied on a value name, which LLVM drops unless --emit=llvm-bc/llvm-ir is given, so the kernel got wrapped twice. I verified on UofT nvidia gpunode (amdgpunodes was not available for my account). r? ZuseZ4

…uwer Rollup of 14 pull requests Successful merges: - #149245 (Make inline attributes apply to the generated `poll` in async fn) - #164304 ([perf] Reuse CanonicalVarKinds if no new variables were added) - #165109 (Avoid building a region map when erasing bound regions) - #163393 (std: Use getexecpath() for current_exe() on OpenBSD) - #164896 (offload: pass the linked device image to the host pass) - #165358 (Reenable some cg_gcc tests that were disabled) - #165371 (Don't use `llvm_type` as the type of `static`s.) - #162232 (Replace `builtin #` with `k#`) - #162608 (Avoid oscillating line wrapping due to symbol indices) - #164241 (make the safety documentation consistent in Ptr module) - #165029 (Use `MalformedLintAttributeInput` in more places) - #165203 (AIX: Implement TLS destructor guard in std and update dynamic-loading-cdylib test) - #165230 (Fix rigidity assertion ICE) - #165374 (Refactorings in preparation for the move trait)

JonathanBrouwer · 2026-11-03T12:29:11Z

💔 I suspect this PR failed tests as part of a rollup
@bors r-

After fixing the problem, consider running a try job for the failed job before re-approving.

Link to failure: #165400 (comment)
@bors try jobs=test-various

rust-bors · 2026-11-03T12:29:20Z

This pull request was unapproved.

This PR was contained in a rollup (#165400), which was unapproved.

View changes since this unapproval

offload: pass the linked device image to the host pass try-job: test-various

rust-bors · 2026-11-03T13:52:38Z

💔 Test for cac7381 failed: CI. Failed job:

try - test-various (web logs, enhanced plaintext logs)

sgasho · 2026-11-04T13:11:28Z

just added needs-offload to ui tests

@bors try jobs=test-various

offload: pass the linked device image to the host pass try-job: test-various

rust-bors · 2026-11-04T14:57:11Z

☀️ Try build successful (CI)
Build commit: 2c2e498 (2c2e498dee8ad78f59a9ce2284ba08d47bb56449)
Base parent: 2d20b42 (2d20b42b883dcfc014f3c2c56081eeb6091c2179)

sgasho · 2026-11-04T15:05:36Z

@bors r=ZuseZ4

rust-bors · 2026-11-04T15:05:41Z

📌 Commit 663a4ba has been approved by ZuseZ4

It is now in the queue for this repository.

rustbot assigned ZuseZ4 Oct 25, 2026

ZuseZ4 added the F-gpu_offload `#![feature(gpu_offload)]` label Oct 25, 2026

This comment has been minimized.

Sign in to view

rust-cloud-vms Bot force-pushed the offload-cleanup-2 branch from 0d7ede9 to 4e185cd Compare October 29, 2026 15:28

ZuseZ4 reviewed Oct 29, 2026

View reviewed changes

Comment thread compiler/rustc_interface/src/passes.rs Resolved

This comment has been minimized.

Sign in to view

rust-cloud-vms Bot force-pushed the offload-cleanup-2 branch from 4e185cd to 4c05263 Compare November 3, 2026 07:00

This comment has been minimized.

Sign in to view

sgasho added 3 commits November 3, 2026 07:04

offload: pass the linked device image to the host pass

e083b8b

offload: make the device build work without --emit=llvm-bc/llvm-ir

b76246d

offload: build bin crates as cdylib on the device side

c6749d8

rust-cloud-vms Bot force-pushed the offload-cleanup-2 branch from 4c05263 to c6749d8 Compare November 3, 2026 07:05

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 3, 2026

JonathanBrouwer mentioned this pull request Nov 3, 2026

Rollup of 14 pull requests #165392

Closed

JonathanBrouwer mentioned this pull request Nov 3, 2026

Rollup of 15 pull requests #165393

Closed

JonathanBrouwer mentioned this pull request Nov 3, 2026

Rollup of 14 pull requests #165394

Closed

JonathanBrouwer mentioned this pull request Nov 3, 2026

Rollup of 14 pull requests #165400

Closed

rust-bors Bot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 3, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Nov 3, 2026

Auto merge of #164896 - sgasho:offload-cleanup-2, r=<try>

cac7381

offload: pass the linked device image to the host pass try-job: test-various

This comment has been minimized.

Sign in to view

add needs-offload to the remaining -Zoffload=Device ui tests

663a4ba

rust-bors Bot pushed a commit that referenced this pull request Nov 4, 2026

Auto merge of #164896 - sgasho:offload-cleanup-2, r=<try>

2c2e498

offload: pass the linked device image to the host pass try-job: test-various

This comment has been minimized.

Sign in to view

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 4, 2026

Conversation

sgasho commented Oct 25, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZuseZ4 commented Oct 25, 2026

Uh oh!

kjetilkjeka commented Oct 26, 2026

Uh oh!

This comment has been minimized.

ZuseZ4 commented Oct 27, 2026

Uh oh!

sgasho commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ZuseZ4 commented Oct 29, 2026

Uh oh!

This comment has been minimized.

ZuseZ4 commented Nov 2, 2026

Uh oh!

rust-bors Bot commented Nov 2, 2026

Uh oh!

ZuseZ4 commented Nov 2, 2026

Uh oh!

rust-bors Bot commented Nov 2, 2026

Uh oh!

ZuseZ4 commented Nov 2, 2026

Uh oh!

rust-bors Bot commented Nov 2, 2026

Uh oh!

This comment has been minimized.

sgasho commented Nov 3, 2026

Uh oh!

rust-bors Bot commented Nov 3, 2026

Uh oh!

JonathanBrouwer commented Nov 3, 2026

Uh oh!

rust-bors Bot commented Nov 3, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Nov 3, 2026

Uh oh!

This comment has been minimized.

sgasho commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

rust-bors Bot commented Nov 4, 2026

Uh oh!

sgasho commented Nov 4, 2026

Uh oh!

rust-bors Bot commented Nov 4, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sgasho commented Oct 25, 2026 •

edited by rustbot

Loading

sgasho commented Oct 29, 2026 •

edited

Loading

sgasho commented Nov 4, 2026 •

edited

Loading