no_std + liballoc can not compile with os based toolchain caused by eh_personality required #106864

Activity

1. gngshn

   added

   C-bugCategory: This is a bug.Category: This is a bug.

   on Jan 14, 2023
2. gngshn commented on Jan 14, 2023

   

   gngshn

   on Jan 14, 2023

   Author

   More actions

   Hi @nbdd0121
   I have filed a new issue

   cc @RalfJung @Amanieu
3. nbdd0121 commented on Jan 14, 2023

   

   nbdd0121

   on Jan 14, 2023

   Member

   More actions

   Here's one potential solution for this:

   • When a panic handler is defined, in addition to rust_begin_unwind, generate a shim function rust_panic_handler_nounwind, which is a mere wrapper of the panic handler, but is defined as #[rustc_nounwind]. This function is generated in the same crate as the panic handler, not libcore.
   • panic_nounwind_fmt then is changed to call this function instead of rust_begin_unwind (i.e. panic_impl lang item).

   This solves the issue here because:

   • panic_nounwind_fmt now does not call into any unwindable function, so it needs no personality.
   • rust_panic_handler_nounwind is generated in the same crate as the panic handler, which will be -Cpanic=abort, so it also does not require personality.

   This approach however does add some complexity to the codegen part.

   Any thoughts, @Amanieu?
4. CAD97 commented on Jan 14, 2023

   

   CAD97

   on Jan 14, 2023

   Contributor

   More actions

   Alternatively (but likely of similar complexity), would it be possible to have panic_nounwind_fmt/#[rustc_nounwind] "just" not introduce a dependency on _rust_eh_personality? Since #[rustc_nounwind] wants to catch and immediately abort on any unwind, it doesn't seem strictly necessary for it to know the Rust unwind personality. It's perfectly fine (and even desirable) for it to just catch/suspend foreign unwinds the same way as Rust unwinds (that is, abort without cleaning up the unwind) rather than inspecting personality to determine if it's a foreign unwind (e.g. to instead print the "cannot catch foreign unwind" rtabort).

   I forget the exact unwinding ABI(s) atm, though, so I don't recall if it's necessary for an unwind terminating frame is still required to specify a personality even if it wants to catch all unwinds including foreign.

   (It'd be good to verify the behavior of #[rustc_nounwind] when hit by a C++ (or other foreign) unwind. Not to say it's necessarily broken, but just to ensure that it isn't.)
5. bjorn3 commented on Jan 14, 2023

   

   bjorn3

   on Jan 14, 2023

   Member

   More actions

   Since #[rustc_nounwind] wants to catch and immediately abort on any unwind, it doesn't seem strictly necessary for it to know the Rust unwind personality.

   The personality function is necessary to catch and abort. Any frame without a personality function is skipped entirely by the unwinder. The abort information is either in a landingpad, which execution is redirected to by the rust personality function or possibly in the future in the LSDA (language specific data area) which is interpreted by the personality function and is an opaque blob to the unwinder.

   It's perfectly fine (and even desirable) for it to just catch/suspend foreign unwinds the same way as Rust unwinds (that is, abort without cleaning up the unwind)

   For rust panics we never call into the unwinder in case of -Cpanic=abort. We literally swap out the function to initiate the rust panic with one which immediately aborts through the panic_abort crate. We can't replace the function to start unwinding for foreign exceptions.

   👍React with 👍1Reacted by Crystal Durham
6. CAD97 commented on Jan 14, 2023

   

   CAD97

   on Jan 14, 2023

   Contributor

   More actions

   So yes, it's not possible to catch/terminate an unwind without specifying a personality. I wasn't certain and thought a catch-all could be defined without, but apparently I misremembered. (It's also possible that's true in some unwind ABIs but not always in general e.g. as exposed by LLVM.)

   aside clarification, off-topic to OP

   And to be clear, I wasn't meaning to say to change anything about how unwinds are started, just how #[rustc_nounwind] terminates unwinds (to prevent the function from unwinding). IIRC extern "C-unwind" immediately catches and aborts on a foreign unwind entering Rust when -Cpanic=abort¹, so a foreign unwind hitting a -Cpanic=abort-mode #[rustc_nounwind] is an impossibility (or at least UB); the aside was off-topic for -Cpanic=unwind² when foreign unwinds can reach it.

   Footnotes

   1. ... Since you say catching unwinds always requires a personality, I assume this means extern "C-unwind" still requires _rust_eh_personality even with -Cpanic=abort -Zbuild-std? That seems at least reasonable (you're explicitly asking to interface with unwinding) but still somewhat unfortunate, since no Rust unwinds will be triggered (on this side of FFI). ↩

   2. Catching foreign unwinds when -Cpanic=unwind is still of course necessary but easy to do when we've got access to the full unwinding runtime. ↩
7. bjorn3 commented on Jan 14, 2023

   

   bjorn3

   on Jan 14, 2023

   Last edited by bjorn3

   Member

   More actions

   I assume this means extern "C-unwind" still requires _rust_eh_personality even with -Cpanic=abort -Zbuild-std?

   For C-unwind -> Rust, yes. For Rust -> C-unwind I believe we skip aborting as no panic/exception can reach it. Or at least we can do so in the future without too much effort I think.

   👍React with 👍1Reacted by Gary Guo
8. gngshn

   changed the title [-]no_std + liballoc can not work again is linux environment[/-] [+]no_std + liballoc can not work again in linux environment[/+] on Jan 15, 2023
9. RalfJung commented on Jan 15, 2023

   

   RalfJung

   on Jan 15, 2023

   Member

   More actions

   Ah, that's unfortunate. :/ Sorry for the breakage.

   We can also revert that PR for now. I won't have the time to do follow-on work here.
10. gngshn commented on Jan 17, 2023

    

    gngshn

    on Jan 17, 2023

    Author

    More actions

    It seems that there is no easy solution to this problem, Since #106045 does not introduce new features or fix any bugs, just behavioral optimization. Can we just revert it first as @RalfJung said, and discuss the consistency of no_std and std panic behavior later when @RalfJung has time to follow on it?
11. bjorn3 commented on Jan 17, 2023

    

    bjorn3

    on Jan 17, 2023

    Member

    More actions

    Only if #102318 is reverted too. Otherwise default_alloc_error_handler will get stabilized in a way that allows unwinding from the alloc error handler, which we wouldn't be able to revert back to aborting by landing #106045 again.
12. gngshn commented on Jan 17, 2023

    

    gngshn

    on Jan 17, 2023

    Last edited by gngshn

    Author

    More actions

    #106045 only change the behavor in no_std environment. And it seems that almost all the no_std crates set panic=abort. Just only revert #106045 seems not bad?
13. bjorn3 commented on Jan 17, 2023

    

    bjorn3

    on Jan 17, 2023

    Member

    More actions

    I guess
14. gngshn commented on Jan 17, 2023

    

    gngshn

    on Jan 17, 2023

    Author

    More actions

    If revert #106045 alone is not an acceptable option, then it is better to continue to wait for someone to fix this problem, and keep these two merged PRs at least allow bare metal development to use the alloc in rust 1.68 stable :)
15. RalfJung commented on Jan 17, 2023

    

    RalfJung

    on Jan 17, 2023

    Member

    More actions

    later when @RalfJung has time to follow on it?

    FWIW I don't think I would do the follow-on in the future either. This was meant to be a "quick fix". I hope someone else can take over. :)
16. 14 remaining items

    Load more

17. nbdd0121 commented on Jan 19, 2023

    

    nbdd0121

    on Jan 19, 2023

    Member

    More actions

    Given Ralf's PR and the default_alloc_error_handler stabilisation PR will be in the same release, technically we are not breaking non-nightly code? Especially as @bjorn3 said it used to work by luck.

    Do we need to revert Ralf's PR for beta, so would it be fine to leave this as is?
18. gngshn commented on Jan 19, 2023

    

    gngshn

    on Jan 19, 2023

    Author

    More actions

    I don't think we need revert Ralf's PR now!
    Looking forward @bjorn3's great work to solve the current compilation problem.
19. bjorn3 commented on Jan 19, 2023

    

    bjorn3

    on Jan 19, 2023

    Member

    More actions

    The solution for which I posted a branch in #106864 (comment) is not something I will finish I think given what I now know about what the actual issue is. I may implement the solution in #106864 (comment) at a later point, but it is probably a bit harder, so I don't think I have time for it right now. I have added it to my backlog, but it might take a while until I get to it. If someone else would like to pick it up that did be great.
20. gngshn commented on Jan 20, 2023

    

    gngshn

    on Jan 20, 2023

    Author

    More actions

    Hi @bjorn3
    Thank you for your support!
    Although it's still not possible to use stable no_std rust in an OS environment, we can do some tests with build-std in nightly rust and cargo!
21. gngshn

    changed the title [-]no_std + liballoc can not work again in linux environment[/-] [+]no_std + liballoc can not compile with os based toolchain caused by eh_personality required[/+] on Jan 20, 2023
22. ChrisDenton

    added

    needs-triage-legacy

    on Jul 16, 2023
23. Noratrieb commented on Oct 19, 2023

    

    Noratrieb

    on Oct 19, 2023

    Member

    More actions

    I'm utterly confused how this didn't produce that error: language item required, but not found: `eh_personality` error for you.

    rust/compiler/rustc_middle/src/middle/lang_items.rs

    Lines 48 to 58 in c3efa51

    	/// Not all lang items are always required for each compilation, particularly in
    	/// the case of panic=abort. In these situations some lang items are injected by
    	/// crates and don't actually need to be defined in libstd.
    	pub fn required(tcx: TyCtxt<'_>, lang_item: LangItem) -> bool {
    	// If we're not compiling with unwinding, we won't actually need these
    	// symbols. Other panic runtimes ensure that the relevant symbols are
    	// available to link things together, but they're never exercised.
    	match tcx.sess.panic_strategy() {
    	PanicStrategy::Abort => {
    	lang_item != LangItem::EhPersonality && lang_item != LangItem::EhCatchTypeinfo
    	}

    apparently this logic thinks that eh_personality is not necessary when using panic=abort.
24. bjorn3 commented on Oct 19, 2023

    

    bjorn3

    on Oct 19, 2023

    Member

    More actions

    If libcore is compiled as panic=abort and extern "C-unwind" isn't used anywhere then eh_personality is indeed not necessary. If any crate is compiled as panic=unwind and extern "C-unwind"` isn't used, then eh_personality isn't reachable, so having rustc produce a dummy impl should be fine.
25. 2moe commented on Dec 16, 2023

    

    2moe

    on Dec 16, 2023

    Last edited by 2moe

    More actions

    I have this issue too.
    The easiest way is to modify the Cargo.toml.

    [profile.release]
    lto = "fat"
    panic = "abort"

    Note: lto = "thin" still has this issue.
    BTW, rustc & cargo version:

    rustc: 1.74.1 (a28077b28 2023-12-04)
    cargo: 1.74.1 (ecb9851af 2023-10-18)
    host: x86_64-unknown-linux-gnu

    ---

    However, during development, I can not always enable Fat LTO + release for compilation speed.
    So I've tried to work on some kinda "hacky" methods.

    #![allow(internal_features)]
    #![cfg_attr(debug_assertions, feature(lang_items))]
    
    #[cfg(debug_assertions)]
    #[lang = "eh_personality"]
    extern "C" fn rust_eh_personality() {}

    It requires nightly toolchain.

    As with thin LTO, some codes work and some don't.

    It works:

    use ::alloc::boxed::Box;
    let b = Box::new("Heap");
    println!("{b}");

    It does not work:

    use libc_print::std_name::println;
    
    let up = "Hello world".to_uppercase();
    println!("{up}");

    ---

    To avoid using the nightly toolchain, and to solve the linking problem.
    I found that simply creating the relevant function manually solved the problem.

    #[no_mangle]
    extern "C" fn rust_eh_personality() {}
    
    #[allow(non_snake_case)]
    #[no_mangle]
    extern "C" fn _Unwind_Resume() {}

    After this, even stable toolchain + dev profile (debug mode) will compile successfully.

    The complete demo code is below:

    #![no_std]
    #![no_main]
    //!
    //! ```cargo
    //! [dependencies]
    //! libc = { version = "0.2.151", default-features = false }
    //! libc-print = "0.1.22"
    //! libc_alloc = "1.0.5"
    //!
    //! [profile.dev]
    //! panic = "abort"
    //!
    //! [profile.release]
    //! # lto = "thin"
    //! panic = "abort"
    //! opt-level = "s"
    //! strip = true
    //! ```
    //!
    #[no_mangle]
    extern "C" fn rust_eh_personality() {}
    
    #[allow(non_snake_case)]
    #[no_mangle]
    extern "C" fn _Unwind_Resume() {}
    // ---------
    
    extern crate alloc;
    use ::alloc::boxed::Box;
    use ::core::{ffi::c_int, panic::PanicInfo};
    use libc_alloc::LibcAlloc;
    use libc_print::std_name::{dbg, eprintln, println};
    
    #[global_allocator]
    static GLOBAL_ALLOC: LibcAlloc = LibcAlloc;
    
    #[panic_handler]
    fn panic(info: &PanicInfo) -> ! {
        eprintln!("[FATAL]: {info}");
        unsafe { libc::abort() }
    }
    
    #[no_mangle]
    pub extern "C" fn main() -> c_int {
        let upper = "ok".to_uppercase();
        println!("{upper}");
    
        let b = Box::new("heap");
        dbg!(b);
        0
    }

    I don't know if it would be a problem to manually create a #[no_mangle] fn _Unwind_Resume(), because it looks really weird.
26. gngshn commented on Dec 18, 2023

    

    gngshn

    on Dec 18, 2023

    Author

    More actions

    I have this issue too. The easiest way is to modify the Cargo.toml.

    [profile.release]
    lto = "fat"
    panic = "abort"

    Note: lto = "thin" still has this issue. BTW, rustc & cargo version:

    rustc: 1.74.1 (a28077b28 2023-12-04)
    cargo: 1.74.1 (ecb9851af 2023-10-18)
    host: x86_64-unknown-linux-gnu

    However, during development, I can not always enable Fat LTO + release for compilation speed. So I've tried to work on some kinda "hacky" methods.

    #![allow(internal_features)]
    #![cfg_attr(debug_assertions, feature(lang_items))]
    
    #[cfg(debug_assertions)]
    #[lang = "eh_personality"]
    extern "C" fn rust_eh_personality() {}

    It requires nightly toolchain.

    As with thin LTO, some codes work and some don't.

    It works:

    use ::alloc::boxed::Box;
    let b = Box::new("Heap");
    println!("{b}");

    It does not work:

    use libc_print::std_name::println;
    
    let up = "Hello world".to_uppercase();
    println!("{up}");

    To avoid using the nightly toolchain, and to solve the linking problem. I found that simply creating the relevant function manually solved the problem.

    #[no_mangle]
    extern "C" fn rust_eh_personality() {}
    
    #[allow(non_snake_case)]
    #[no_mangle]
    extern "C" fn _Unwind_Resume() {}

    After this, even stable toolchain + dev profile (debug mode) will compile successfully.

    The complete demo code is below:

    #![no_std]
    #![no_main]
    //!
    //! ```cargo
    //! [dependencies]
    //! libc = { version = "0.2.151", default-features = false }
    //! libc-print = "0.1.22"
    //! libc_alloc = "1.0.5"
    //!
    //! [profile.dev]
    //! panic = "abort"
    //!
    //! [profile.release]
    //! # lto = "thin"
    //! panic = "abort"
    //! opt-level = "s"
    //! strip = true
    //! ```
    //!
    #[no_mangle]
    extern "C" fn rust_eh_personality() {}
    
    #[allow(non_snake_case)]
    #[no_mangle]
    extern "C" fn _Unwind_Resume() {}
    // ---------
    
    extern crate alloc;
    use ::alloc::boxed::Box;
    use ::core::{ffi::c_int, panic::PanicInfo};
    use libc_alloc::LibcAlloc;
    use libc_print::std_name::{dbg, eprintln, println};
    
    #[global_allocator]
    static GLOBAL_ALLOC: LibcAlloc = LibcAlloc;
    
    #[panic_handler]
    fn panic(info: &PanicInfo) -> ! {
        eprintln!("[FATAL]: {info}");
        unsafe { libc::abort() }
    }
    
    #[no_mangle]
    pub extern "C" fn main() -> c_int {
        let upper = "ok".to_uppercase();
        println!("{upper}");
    
        let b = Box::new("heap");
        dbg!(b);
        0
    }

    I don't know if it would be a problem to manually create a #[no_mangle] fn _Unwind_Resume(), because it looks really weird.

    Thanks for your infomation. This seems another good way to test no_std + liballoc on stable toolchain.

    As you said, doing this seems to replace the default implementation of _Unwind_Resume and rust_eh_personality in rust static lib. This also can not be used in production environment. It maybe UB? And the naming _Unwind_Resume may be changed in the future?

    We also can use cargo build std with RUSTC_BOOTSTRAP=1, But I think this is also not a real safty way and can not be used in production environment.
27. bjorn3 commented on Dec 18, 2023

    

    bjorn3

    on Dec 18, 2023

    Last edited by bjorn3

    Member

    More actions

    And the naming _Unwind_Resume may be changed in the future?

    Nope, that is a stable interface of the itanium style unwinding that is used on almost all Unix systems.

    It maybe UB?

    If your program is linking in C++ code that wants to throw exceptions, yes. But in that case you would link against libunwind anyway and thus not have this undefined reference.

    As for rust_eh_personality, that one is indeed unstable though.

    By the way all targets that are meant exclusively for no_std usage don't have this issue as the standard library is already compiled with panic=abort. It is a real problem for the targets that are for running under a conventional OS though.

    👍React with 👍1Reacted by gngshn
28. GrigorenkoPV commented on Feb 22, 2024

    

    GrigorenkoPV

    on Feb 22, 2024

    Contributor

    More actions

    Might be related to #56152
29. jieyouxu

    added

    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.Relevant to the compiler team, which will review and decide on the PR/issue.

    A-linkageArea: linking into static, shared libraries and binariesArea: linking into static, shared libraries and binaries

    and removed

    needs-triage-legacy

    on Nov 12, 2024