Repository navigation
Rust 1.59 using linux-musl and PIE #94364
Description
I'm trying to upgrade our codebase to rust 1.59.0. It compiles fine on 1.58.0.
For linux targets, we compile using the x86_64-unknown-linux-musl target.
But with 1.59.0, I'm getting many such errors
relocation R_X86_64_32 against `.bss.internal_state.3389' can not be used when making a PIE object; recompile with -fPIE
Followed by this
final link failed: nonrepresentable section on output
I'm guessing this has to do with this change.
I tried compiling with both -C relocation-model=pie and relocation-model=static but it did not seem to change much.
Any advice?
Activity
apiraino commentedon Feb 26, 2022
@amircodota I'm figuring out if your issue it reproducible with something as minimal as a hello world, something like :
(if it helps to diagnose the issue)
amircodota commentedon Feb 26, 2022
@apiraino I seriously doubt it.
This is a pretty complex multi-crate workspace, with many dependencies, some of which contain c and c++ code.
I'm pasting a full output of the error - maybe ot would help zero in of the issue
amircodota commentedon Feb 26, 2022
@apiraino What I am really looking for is a workaround.
I think 1.59 changed the default relocation-model of musl.
If you guys can provide instructions on how to set the relevant flags so that it behaves the same as 1.58, I think that would be good enough :-)
Amanieu commentedon Feb 26, 2022
From the error message it seems that musl itself is not compiled with
-fPIE. You are using a custom/usr/local/x86_64-linux-musl/lib/libc.ainstead of the one that is provided by rustup.amircodota commentedon Feb 26, 2022
hmmm... Interesting.
I am using cross to compile musl, and I have a custom Dockerfile, that is built from the cross base docker image for musl.
Let me see if I can reproduce on simple code using cross
amircodota commentedon Feb 26, 2022
using cross on a brand new
cargo initcode, it does not fail, but it gives a warningMaybe this would give a clue?
Here's how to reproduce
Amanieu commentedon Feb 26, 2022
You don't need to use cross to target musl, you can just install the target using
rustup target addand then usecargo builddirectly.If this works then you should report a bug in cross instead.
12101111 commentedon Feb 26, 2022
I can't reproduce this in cross,
It use a old gcc 6.4.0 and pick musl's libc.a from rust, your are using gcc 9.2 and pick musl's libc.a from gcc:
amircodota commentedon Feb 26, 2022
Yes, this is indeed something with my custom cross docker image.
I'm investigating on my end.
amircodota commentedon Feb 26, 2022
Posting an update here - in case someone else runs into this issue too.
I was using a custom cross docker image which added some depndencies to the cross base image.
In the Dockerfile I had the following line
The reason was one of the crates I was using needed libstdc++.a, which does not come bundled with the rust target, but does exist in the above directory in the cross docker image.
This worked fine until rust 1.59.0, where the musl target changed to use PIE by default, and so my docker image was no longer working.
The solution was simple - just copy libstddc++.a to a different directory, and add that directory to the linker search path.
Thanks everyone for the help
horacimacias commentedon Mar 12, 2022
I think I'm having a similar issue, although I don't see any
libstdc++.afile under my musl folders. @amircodota , would you mind sharing some bits of your custom dockerfile or at least the part where musl/libstdc++.a are involved? perhaps this helps me fix my issue.amircodota commentedon Mar 12, 2022
@horacimacias I'll try to exaplin in a little more detail what was the issue for me - hopefully this can help you figure out your case.
In my case I had a custom docker image, based off
rustembedded/cross:x86_64-unknown-linux-musl.This image already comes bundled with musl static and shared libraries under
/usr/local/x86_64-linux-musl/lib.However, by default, the rust compiler will not need those libraries, because it has the libraries it needs under the rust musl target directory.
In my case, I needed to link against some static libraries that were not included in with the rust musl target (in my case it was libstdc++.a, but there are other libraries that might be relevant).
To fix that, I added the following environment variable to my custom dockerfile
This worked fine until rust 1.59.0, where the musl target is compiled with different flags, causing linkage errors.
So to fix it, I copied the libraries I needed from
/usr/local/x86_64-linux-musl/libinto a different dicrectory, and set that other directory in the linker search pathThe bottom line is that you want to make sure you link against the musl libc that comes bundled with your rust target, and not with some other musl libc that happens to exist somewhere in your docker image.
Hope this helps :-)
horacimacias commentedon Mar 13, 2022
it does help, thanks.
In my case my image is based on okteto and on top of that we're installing libraries for other things (e.g. libpq, openssl).
In the end we were missing some
-Lentries to rustflags as well. After adding them we seem to be good now.thanks!
brocaar commentedon May 25, 2023
Thanks @amircodota I have been pulling my hairs out for almost a complete day and I couldn't figure out the issue. Your hint was really helpful. In my case I was using a custom cross image as well and I was installing additional libraries into the
/usr/local/x86_64-linux-muslprefix.Instead of copying the files, I decided to use a different prefix when compiling these libraries (openssl, zlib and pq). I think this might be easier, because this will also set the correct paths in the
pkgconfig/*.pcfiles.Here is an example that compiles fine, maybe it is useful for others when they experience the same and find this GitHub issue:
(The Protobuf compiler is actually not needed in this example, but I will need it for my code later on).
Cargo.toml:main.rs:nix: revert to -static flag for aarch64 compatibility
nix: revert to -static flag for aarch64 compatibility
nix: revert to -static flag for aarch64 compatibility
nix: update comment about -static flag for aarch64 compatibility