Repository navigation
Conversation
bjorn3
added
A-metadata
Contributor
|
r? @davidtwco (rust-highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
Member
Author
|
@bors try @rust-timer queue |
Collaborator
|
Awaiting bors try build completion. @rustbot label: +S-waiting-on-perf |
rustbot
added
the
S-waiting-on-perf
Collaborator
|
⌛ Trying commit bf8a8f82a73414610384b4e36323c0966db8e908 with merge 86e45ec336f0ccde044af1ace8b3c425239d01f2... |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Collaborator
|
💔 Test failed - checks-actions |
bors
added
S-waiting-on-author
fbstj
reviewed
Feb 12, 2022
Comment on lines
549
to
556
Contributor
There was a problem hiding this comment.
this comment doesn't make sense with the code below it
Member
Author
There was a problem hiding this comment.
I'm not sure where it came from. Maybe it was correct in an old draft? I finished a draft from September last year just now.
Dylib metadata compression increases code complexity and will become less important once -Zsplit-metadata is introduced
I don't think it is necessary anymore. As I understand it from issue 39504 the original problem was that rustbuild changed a hardlink in the cargo build dir to point to copy in the sysroot while cargo may have hardlinked it to the original first. I don't think this happens anymore and as such this workaround is no longer necessary.
If a crate is both an rlib and cdylib, the metadata loader would previously incorrectly assume that it is possible to link against the cdylib. Cdylibs don't export all symbols required for linking as rust library. Possibly fixes issue number 82151
This will make it easier to report rustc versions for older metadata formats.
This will split the crate metadata out of library files. Instead only the svh is preserved to allow for loading the right rmeta file. This significicantly reduces library size. In addition it allows for cheaper checks if different library files are the same crate.
bjorn3
force-pushed
the
split_metadata3
branch
from
February 12, 2022 18:20
bf8a8f8 to
af25358
Compare
Member
Author
|
@bors try @rust-timer queue |
Collaborator
|
Awaiting bors try build completion. @rustbot label: +S-waiting-on-perf |
Collaborator
|
⌛ Trying commit af25358 with merge 3e62ae8cce9be671b29a9715b3d390e4ad0c92d2... |
This comment has been minimized.
This comment has been minimized.
bjorn3
force-pushed
the
split_metadata3
branch
from
February 13, 2022 11:34
6a21ba1 to
c0d8f3e
Compare
Collaborator
|
The job Click to see the possible cause of the failure (guessed by this bot) |
bjorn3
marked this pull request as draft
February 13, 2022 11:50
Member
Author
|
I think what happens is that when |
petrochenkov
added
S-waiting-on-review
Collaborator
|
☔ The latest upstream changes (presumably #94103) made this pull request unmergeable. Please resolve the merge conflicts. |
Member
|
closing this due to inactivity |
Dylan-DPC
added
S-inactive
This was referenced May 15, 2022
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Feb 9, 2024
Introduce -Zsplit-metadata option This will split the crate metadata out of library files. Instead only the svh and a bit of extra metadata is preserved to allow for loading the right rmeta file. This significantly reduces library size. In addition it allows for cheaper checks if different library files are the same crate. A fair amount of the complexity in this PR is to work around the fact that cargo doesn't directly support this option yet. Fixes rust-lang#23366 Fixes rust-lang#57076 Revives rust-lang#93945
Member
Author
|
Opened #120855 to revive this. |
Dylan-DPC
removed
the
S-inactive
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Apr 1, 2025
Introduce `-Zembed-metadata` to allow omitting full metadata from rlibs and dylibs This is a continuation of rust-lang#120855 (I was mentored by `@bjorn3` to move it forward). Most of the original code was written by bjorn3, I tried to clean it up a bit and add some documentation and tests. This PR introduces a new unstable compiler flag called `-Zembed-metadata=[no|yes]`, with the default being `yes` (see rust-lang#57076 for context). When set to `no`, rustc will only store a small metadata stub inside rlibs/dylibs instead of the full metadata, to keep their size smaller. It should be used in combination with `--emit=metadata`, so that the users of such a compiled library can still read the metadata from the corresponding `.rmeta` file. [This comment](rust-lang#120855 (comment)) shows an example of binary/artifact size wins that can be achieved using this approach. Contrary to rust-lang#120855, this PR only introduces the new flag, along with a couple of run-make tests and documentation, but does not yet use it in bootstrap to actually compile rustc. I plan to do that as a follow-up step (along with integration in Cargo, which should ideally just always pass this flag to reduce the size of target directories). Fixes rust-lang#23366 Closes rust-lang#29511 Fixes rust-lang#57076 Another attempt of rust-lang#93945 and rust-lang#120855. r? `@petrochenkov`
lnicola
pushed a commit
to lnicola/rust-analyzer
that referenced
this pull request
Apr 28, 2025
Introduce `-Zembed-metadata` to allow omitting full metadata from rlibs and dylibs This is a continuation of rust-lang/rust#120855 (I was mentored by `@bjorn3` to move it forward). Most of the original code was written by bjorn3, I tried to clean it up a bit and add some documentation and tests. This PR introduces a new unstable compiler flag called `-Zembed-metadata=[no|yes]`, with the default being `yes` (see rust-lang/rust#57076 for context). When set to `no`, rustc will only store a small metadata stub inside rlibs/dylibs instead of the full metadata, to keep their size smaller. It should be used in combination with `--emit=metadata`, so that the users of such a compiled library can still read the metadata from the corresponding `.rmeta` file. [This comment](rust-lang/rust#120855 (comment)) shows an example of binary/artifact size wins that can be achieved using this approach. Contrary to rust-lang/rust#120855, this PR only introduces the new flag, along with a couple of run-make tests and documentation, but does not yet use it in bootstrap to actually compile rustc. I plan to do that as a follow-up step (along with integration in Cargo, which should ideally just always pass this flag to reduce the size of target directories). Fixes rust-lang/rust#23366 Closes rust-lang/rust#29511 Fixes rust-lang/rust#57076 Another attempt of rust-lang/rust#93945 and rust-lang/rust#120855. r? `@petrochenkov`
JonathanBrouwer
added a commit
to JonathanBrouwer/rust
that referenced
this pull request
Oct 19, 2026
…=petrochenkov Stabilize `-Zembed-metadata` Tracking issue: rust-lang#139165 This PR proposes to stabilize the `-Zembed-metadata` compiler flag, so that Cargo can start using `-Cembed-metadata=no` by default on the stable channel. The flag can be used to avoid duplicating the metadata of Rust crates within `.rlib` and `.rmeta` files. This can reduce the size of the `target` directory anywhere from 5% to 35% on real-world projects, depending on the chosen compilation profile ([1][kobzol-blog], [2][cargo-blog]). Additionally, `-Cembed-metadata=no` also removes the metadata out of Rust `dylib`s, and thus reduces their size. The plan is to stabilize this compiler flag so that Cargo can start using it on the stable channel. The stabilization plan thus has two parts: - Stabilize the `-Cembed-metadata` flag in `rustc` - Change Cargo to use `-Cembed-metadata=no` by default The Cargo team has prepared their side of the [stabilization report](rust-lang/cargo#17533). You can examine it to see the whole picture. ## What is being stabilized The `-Zembed-metadata` compiler flag. Note that `rustc` will still default to `-Cembed-metadata=yes`, as that is its original behavior since [Rust 1.16.0](https://blog.rust-lang.org/2017/03/16/Rust-1.16/#other-improvements). But it will now be possible to pass `-Cembed-metadata=no` on stable. ## Motivation When compiling Rust crates as libraries, Cargo uses pipelined compilation, which tells `rustc` (using `--emit=metadata`) to output an `.rmeta` file as soon as possible, so that Cargo can start follow-up `rustc` instances sooner and speed up the compilation process. This file contains the Rust-specific metadata of the compiled crate. However, after the compilation of the Rust crate finishes, the compiler will also write an `.rlib` file to disk, which contains both the compiled object code, but also the same metadata that is already contained in the `.rmeta` file. This means that after the compilation finishes, the metadata of each (non-leaf) compiled crate will be duplicated on disk. The metadata can be quite large, and it is not uncommon that it takes hundreds of megabytes for real-world projects. This duplication thus contributes to increasing the size of the `target` directory, which large size is one of the top 3 most pressing [challenges](https://blog.rust-lang.org/2026/03/02/2025-State-Of-Rust-Survey-results/#challenges-and-wishes-about-rust) that Rust users struggle with. The `-Cembed-metadata=no` flag removes this duplication (see [below](#What-does-the-flag-do) on how it works). Some size benchmark results can be found in ([1][kobzol-blog], [2][cargo-blog]). It can reach up to 35% size reduction on real-world projects in `release` mode, with the effect being smaller for `dev` builds (because incremental artifacts and debuginfo start to dominate the `target` directory size). A slight bonus is that we only write the metadata to disk once, thus resulting in less I/O. I haven't noticed any measurable effect (either positive or negative) on compile times though, at least on Linux. The main motivation is disk size, not compilation speed. ## What does the flag do When you pass `-Cembed-metadata=no`, `rustc` will not write the full metadata of the compiled crate into the emitted `.rlib` file. Instead, it will only write a ["metadata stub"](https://github.com/rust-lang/rust/blob/79ee8dc08073a0abf93d2b494ac5c1be11d4342e/compiler/rustc_metadata/src/rmeta/encoder.rs#L2552), which only contains the most basic information (name, target tuple, SVH). Note that this flag should usually be combined with `--emit=metadata` to write the metadata to a `.rmeta` file, otherwise no metadata would be generated at all. When loading crate dependencies passed via `--extern`, `rustc` will read the metadata of the passed `.rlib`. If it only contains a stub, it expects that you will also pass another `--extern` flag with a path to a `.rmeta` file with the full metadata. If you don't, you will be met with the following error: ``` error: only metadata stub found for `rlib` dependency `foo`. Please provide path to the corresponding .rmeta file with full metadata ``` Note that `rustc` will still default to the original behavior of embedding metadata, so build systems that wrap `rustc` will have to explicitly opt into using `-Cembed-metadata=no`, and at that point they will also have to start passing `--extern=<...>.rmeta` flags to `rustc`. That is what Cargo currently does if you use `nightly` or if you use the unstable `-Zembed-metadata=no` *Cargo* flag. In theory, we could relax the requirement to pass both paths to the `.rlib` and the `.rmeta` file for each direct dependency in the future, and let `rustc --extern=foo/bar.rlib` automatically attempt to load `--extern=foo/bar.rmeta` when the `.rmeta` file path is not passed explicitly. `.rlib` and `.rmeta` files produced using `-Cembed-metadata=no` and `-Cembed-metadata=yes` can be freely combined, and a downstream `rustc` invocation that consumes artifacts produced by `=no` does not need to itself receive the `=no` flag. ## Ecosystem impact The `-Zembed-metadata` flag has been used for the standard library that we ship since January 2026 (rust-lang#145343) and for the compiler itself and codegen backends (cg_clif and cg_gcc) since August 2026 (rust-lang#151061). The flag has also been used by Cargo by default on the `nightly` channel since the second half of August 2026. A blog post with a call for testing can be found [here](https://blog.rust-lang.org/inside-rust/2026/08/18/reducing-target-dir-size-on-nightly/). I'm not aware of any regressions or issues reported in relation to this, not is the Cargo team (AFAIK). There are some unresolved questions about backwards compability if Cargo starts passing `-Cembed-metadata=no` by default, but those are related strictly to Cargo, and not the compiler flag itself (because the compiler flag will remain opt-in). ## History The issue of duplicated metadata was originally noted back in 2015 (rust-lang#23366, rust-lang#29511), back then because of the metadata being duplicated in Rust `dylib`s. Since Cargo started using pipelined compilation by default, it became more acute, because the metadata of all compiled `.rlib` crate dependencies started being duplicated in the `target` directory. @bjorn3 proposed to add a compiler flag to split out the metadata into a separate file in rust-lang#57076. The implementation was attempted in rust-lang#93945 and later in rust-lang#120855, but there was non-trivial complexity due to Cargo integration. In 2025, I opened an [MCP](rust-lang/compiler-team#851) with the plan to implement this flag in two stages (first in rustc, and only then in Cargo), to reduce the implementation complexity, which was accepted. The implementation in `rustc` then happened in rust-lang#137535. Implementation history: - rust-lang#137535 (April 2025): Introduction of `-Zembed-metadata`, reviewed by @petrochenkov - rust-lang/cargo#15378 (May 2025): Initial integration in Cargo, reviewed by @epage - rust-lang#145343 (January 2026): Dogfood the flag for stdlib, reviewed by @bjorn3 - rust-lang#151061 (August 2026): Dogfood the flag for rustc and codegen backends, authored by @bjorn3 - rust-lang/cargo#17267 (August 2026): Enabled the `-Cembed-metadata=no` flag by default by Cargo on the `nightly` channel ## Unresolved questions/concerns ### Command-line length limit This is more related to the Cargo side of things, but I also wanted to mention it here, because there is a possible improvement we could do on the compiler side in the future. As noted [above](#What-does-the-flag-do), usage of `-Cembed-metadata=no` requires the build system/user to pass two `--extern` flags for each direct dependency of a given `rustc` invocation. This can increase the size of the executed command-line arguments, which can in edge cases start running into operating system limits. Recently, we saw something similar in action with the Cargo's new build dir layout stabilization, where Cargo suddenly started passing a lot more `-L` flags to `rustc`, which caused issues around CLI length limits and `PATH` env. var. length limits. Cargo can work around the CLI length limit by using response files (`rustc @args`), but not all downstream tools (such as `sccache`) support those fully (yet). Based on the experience with the new build dir layout, some work was done to improve this support in the ecosystem though, and to optimize the number of flags that Cargo passes to `rustc`. If this became a problem in practice, we could teach `rustc` to start implicitly assuming `--extern=foo.rmeta` when `--extern=foo.rlib` was passed. This would reduce the number of required `--extern` flags passed to the same count as with `-Cembed-metadata=yes` (or without using that flag at all). I think that we can always do this retroactively if needed without breaking backwards compatibility, so it does not have to block this stabilization. ### Landing this change Given that this compiler flag is expected to have tight integration with Cargo, and Cargo already uses it on the nightly channel by default, we have to be a bit careful about how we actually land the `-Zembed-metadata` => `-Cembed-metadata` change. Problem A: if we simply switched it from `-Z` to `-C` in `rust-lang/rust`, then nightly would break from the following day, because Cargo would be still attempting to pass the `-Z` flag, while `rustc` would already be expecting the `-C` variant of the flag. Problem B: furthermore, bootstrap currently unconditionally passes `-Zembed-metadata=no`, both to in-tree Cargo and to beta Cargo, so that also requires some attention. There are several options of how we could resolve this: 1. Conclude the nightly Cargo experiment, so that it will not pass `-Zembed-metadata=no` to `rustc` by default anymore. Then sync Cargo to `rust-lang/rust`, and only then merge this PR. This solves A. 2. Move Cargo from a git submodule to a (Josh) subtree, so that we can atomically change both Cargo and `rustc` to use `-C` instead of `-Z` (inside this PR), to avoid breaking nightly users. This solves A. - Even though I would like to do this migration anyway, I don't think that we need to tie it to this stabilization, as it can take some time before we resolve all concerns about the migration. 3. Do what 2. proposes, but without a subtree. That means that we have to: - Change to `-C` in Cargo - Change to `-C` in `rust-lang/rust` - Sync the Cargo change back to `rust-lang/rust` in the same day that we land the `rustc` change While this solves A, it might run into some staging issues in bootstrap, and it looks very complicated to pull off. 4. Teach `rustc` to accept both the `-Z` and the `-C` variant of the flag for some time. This solves both A and B, and also gives people who use `-Zembed-metadata` manually on `nightly` some breathing room to upgrade to the `-C` variant of the flag (but GitHub code search didn't find many usages of this flag, and this is normally an expected breakage for users of unstable compiler flags). I decided to go with 4, because the integration with Cargo is tight, and modifying bootstrap back and forth to support `-Z`/no flag /`-C` would be busy work. This solves both A and B and requires little implementation effort. ## Acknowledgments - @bjorn3 for coming up with the idea, providing a reference implementation and mentoring me to finish up the implementation - @petrochenkov for reviewing the compiler side of things - @epage and @weihanglo for reviewing the Cargo side of things [kobzol-blog]: https://kobzol.github.io/rust/rustc/2025/06/02/reduce-cargo-target-dir-size-with-z-no-embed-metadata.html#cargo-integration [cargo-blog]: https://blog.rust-lang.org/inside-rust/2026/08/18/reducing-target-dir-size-on-nightly/#how-much-does-it-help CC @bjorn3 @petrochenkov @weihanglo @epage r? petrochenkov @rustbot label +needs-fcp +T-compiler
rust-bors Bot
pushed a commit
that referenced
this pull request
Oct 19, 2026
Rollup merge of #163436 - Kobzol:stabilize-embed-metadata, r=petrochenkov Stabilize `-Zembed-metadata` Tracking issue: #139165 This PR proposes to stabilize the `-Zembed-metadata` compiler flag, so that Cargo can start using `-Cembed-metadata=no` by default on the stable channel. The flag can be used to avoid duplicating the metadata of Rust crates within `.rlib` and `.rmeta` files. This can reduce the size of the `target` directory anywhere from 5% to 35% on real-world projects, depending on the chosen compilation profile ([1][kobzol-blog], [2][cargo-blog]). Additionally, `-Cembed-metadata=no` also removes the metadata out of Rust `dylib`s, and thus reduces their size. The plan is to stabilize this compiler flag so that Cargo can start using it on the stable channel. The stabilization plan thus has two parts: - Stabilize the `-Cembed-metadata` flag in `rustc` - Change Cargo to use `-Cembed-metadata=no` by default The Cargo team has prepared their side of the [stabilization report](rust-lang/cargo#17533). You can examine it to see the whole picture. ## What is being stabilized The `-Zembed-metadata` compiler flag. Note that `rustc` will still default to `-Cembed-metadata=yes`, as that is its original behavior since [Rust 1.16.0](https://blog.rust-lang.org/2017/03/16/Rust-1.16/#other-improvements). But it will now be possible to pass `-Cembed-metadata=no` on stable. ## Motivation When compiling Rust crates as libraries, Cargo uses pipelined compilation, which tells `rustc` (using `--emit=metadata`) to output an `.rmeta` file as soon as possible, so that Cargo can start follow-up `rustc` instances sooner and speed up the compilation process. This file contains the Rust-specific metadata of the compiled crate. However, after the compilation of the Rust crate finishes, the compiler will also write an `.rlib` file to disk, which contains both the compiled object code, but also the same metadata that is already contained in the `.rmeta` file. This means that after the compilation finishes, the metadata of each (non-leaf) compiled crate will be duplicated on disk. The metadata can be quite large, and it is not uncommon that it takes hundreds of megabytes for real-world projects. This duplication thus contributes to increasing the size of the `target` directory, which large size is one of the top 3 most pressing [challenges](https://blog.rust-lang.org/2026/03/02/2025-State-Of-Rust-Survey-results/#challenges-and-wishes-about-rust) that Rust users struggle with. The `-Cembed-metadata=no` flag removes this duplication (see [below](#What-does-the-flag-do) on how it works). Some size benchmark results can be found in ([1][kobzol-blog], [2][cargo-blog]). It can reach up to 35% size reduction on real-world projects in `release` mode, with the effect being smaller for `dev` builds (because incremental artifacts and debuginfo start to dominate the `target` directory size). A slight bonus is that we only write the metadata to disk once, thus resulting in less I/O. I haven't noticed any measurable effect (either positive or negative) on compile times though, at least on Linux. The main motivation is disk size, not compilation speed. ## What does the flag do When you pass `-Cembed-metadata=no`, `rustc` will not write the full metadata of the compiled crate into the emitted `.rlib` file. Instead, it will only write a ["metadata stub"](https://github.com/rust-lang/rust/blob/79ee8dc08073a0abf93d2b494ac5c1be11d4342e/compiler/rustc_metadata/src/rmeta/encoder.rs#L2552), which only contains the most basic information (name, target tuple, SVH). Note that this flag should usually be combined with `--emit=metadata` to write the metadata to a `.rmeta` file, otherwise no metadata would be generated at all. When loading crate dependencies passed via `--extern`, `rustc` will read the metadata of the passed `.rlib`. If it only contains a stub, it expects that you will also pass another `--extern` flag with a path to a `.rmeta` file with the full metadata. If you don't, you will be met with the following error: ``` error: only metadata stub found for `rlib` dependency `foo`. Please provide path to the corresponding .rmeta file with full metadata ``` Note that `rustc` will still default to the original behavior of embedding metadata, so build systems that wrap `rustc` will have to explicitly opt into using `-Cembed-metadata=no`, and at that point they will also have to start passing `--extern=<...>.rmeta` flags to `rustc`. That is what Cargo currently does if you use `nightly` or if you use the unstable `-Zembed-metadata=no` *Cargo* flag. In theory, we could relax the requirement to pass both paths to the `.rlib` and the `.rmeta` file for each direct dependency in the future, and let `rustc --extern=foo/bar.rlib` automatically attempt to load `--extern=foo/bar.rmeta` when the `.rmeta` file path is not passed explicitly. `.rlib` and `.rmeta` files produced using `-Cembed-metadata=no` and `-Cembed-metadata=yes` can be freely combined, and a downstream `rustc` invocation that consumes artifacts produced by `=no` does not need to itself receive the `=no` flag. ## Ecosystem impact The `-Zembed-metadata` flag has been used for the standard library that we ship since January 2026 (#145343) and for the compiler itself and codegen backends (cg_clif and cg_gcc) since August 2026 (#151061). The flag has also been used by Cargo by default on the `nightly` channel since the second half of August 2026. A blog post with a call for testing can be found [here](https://blog.rust-lang.org/inside-rust/2026/08/18/reducing-target-dir-size-on-nightly/). I'm not aware of any regressions or issues reported in relation to this, not is the Cargo team (AFAIK). There are some unresolved questions about backwards compability if Cargo starts passing `-Cembed-metadata=no` by default, but those are related strictly to Cargo, and not the compiler flag itself (because the compiler flag will remain opt-in). ## History The issue of duplicated metadata was originally noted back in 2015 (#23366, #29511), back then because of the metadata being duplicated in Rust `dylib`s. Since Cargo started using pipelined compilation by default, it became more acute, because the metadata of all compiled `.rlib` crate dependencies started being duplicated in the `target` directory. @bjorn3 proposed to add a compiler flag to split out the metadata into a separate file in #57076. The implementation was attempted in #93945 and later in #120855, but there was non-trivial complexity due to Cargo integration. In 2025, I opened an [MCP](rust-lang/compiler-team#851) with the plan to implement this flag in two stages (first in rustc, and only then in Cargo), to reduce the implementation complexity, which was accepted. The implementation in `rustc` then happened in #137535. Implementation history: - #137535 (April 2025): Introduction of `-Zembed-metadata`, reviewed by @petrochenkov - rust-lang/cargo#15378 (May 2025): Initial integration in Cargo, reviewed by @epage - #145343 (January 2026): Dogfood the flag for stdlib, reviewed by @bjorn3 - #151061 (August 2026): Dogfood the flag for rustc and codegen backends, authored by @bjorn3 - rust-lang/cargo#17267 (August 2026): Enabled the `-Cembed-metadata=no` flag by default by Cargo on the `nightly` channel ## Unresolved questions/concerns ### Command-line length limit This is more related to the Cargo side of things, but I also wanted to mention it here, because there is a possible improvement we could do on the compiler side in the future. As noted [above](#What-does-the-flag-do), usage of `-Cembed-metadata=no` requires the build system/user to pass two `--extern` flags for each direct dependency of a given `rustc` invocation. This can increase the size of the executed command-line arguments, which can in edge cases start running into operating system limits. Recently, we saw something similar in action with the Cargo's new build dir layout stabilization, where Cargo suddenly started passing a lot more `-L` flags to `rustc`, which caused issues around CLI length limits and `PATH` env. var. length limits. Cargo can work around the CLI length limit by using response files (`rustc @args`), but not all downstream tools (such as `sccache`) support those fully (yet). Based on the experience with the new build dir layout, some work was done to improve this support in the ecosystem though, and to optimize the number of flags that Cargo passes to `rustc`. If this became a problem in practice, we could teach `rustc` to start implicitly assuming `--extern=foo.rmeta` when `--extern=foo.rlib` was passed. This would reduce the number of required `--extern` flags passed to the same count as with `-Cembed-metadata=yes` (or without using that flag at all). I think that we can always do this retroactively if needed without breaking backwards compatibility, so it does not have to block this stabilization. ### Landing this change Given that this compiler flag is expected to have tight integration with Cargo, and Cargo already uses it on the nightly channel by default, we have to be a bit careful about how we actually land the `-Zembed-metadata` => `-Cembed-metadata` change. Problem A: if we simply switched it from `-Z` to `-C` in `rust-lang/rust`, then nightly would break from the following day, because Cargo would be still attempting to pass the `-Z` flag, while `rustc` would already be expecting the `-C` variant of the flag. Problem B: furthermore, bootstrap currently unconditionally passes `-Zembed-metadata=no`, both to in-tree Cargo and to beta Cargo, so that also requires some attention. There are several options of how we could resolve this: 1. Conclude the nightly Cargo experiment, so that it will not pass `-Zembed-metadata=no` to `rustc` by default anymore. Then sync Cargo to `rust-lang/rust`, and only then merge this PR. This solves A. 2. Move Cargo from a git submodule to a (Josh) subtree, so that we can atomically change both Cargo and `rustc` to use `-C` instead of `-Z` (inside this PR), to avoid breaking nightly users. This solves A. - Even though I would like to do this migration anyway, I don't think that we need to tie it to this stabilization, as it can take some time before we resolve all concerns about the migration. 3. Do what 2. proposes, but without a subtree. That means that we have to: - Change to `-C` in Cargo - Change to `-C` in `rust-lang/rust` - Sync the Cargo change back to `rust-lang/rust` in the same day that we land the `rustc` change While this solves A, it might run into some staging issues in bootstrap, and it looks very complicated to pull off. 4. Teach `rustc` to accept both the `-Z` and the `-C` variant of the flag for some time. This solves both A and B, and also gives people who use `-Zembed-metadata` manually on `nightly` some breathing room to upgrade to the `-C` variant of the flag (but GitHub code search didn't find many usages of this flag, and this is normally an expected breakage for users of unstable compiler flags). I decided to go with 4, because the integration with Cargo is tight, and modifying bootstrap back and forth to support `-Z`/no flag /`-C` would be busy work. This solves both A and B and requires little implementation effort. ## Acknowledgments - @bjorn3 for coming up with the idea, providing a reference implementation and mentoring me to finish up the implementation - @petrochenkov for reviewing the compiler side of things - @epage and @weihanglo for reviewing the Cargo side of things [kobzol-blog]: https://kobzol.github.io/rust/rustc/2025/06/02/reduce-cargo-target-dir-size-with-z-no-embed-metadata.html#cargo-integration [cargo-blog]: https://blog.rust-lang.org/inside-rust/2026/08/18/reducing-target-dir-size-on-nightly/#how-much-does-it-help CC @bjorn3 @petrochenkov @weihanglo @epage r? petrochenkov @rustbot label +needs-fcp +T-compiler
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This will split the crate metadata out of library files. Instead only the svh is preserved to allow for loading the right rmeta file. This significicantly reduces library size. In addition it allows for cheaper checks if different library files are the same crate.
Fixes #23366
Fixes #57076
Fixes #10786 (through 453d9a0bf778e385b45ecced681a71315fe8517e)
Should fix #82151 (through 453d9a0bf778e385b45ecced681a71315fe8517e, unconfirmed)