Emit thin-link-bitcode causes ThinLTOBitcodeWriterPass to run twice #164172

I think I've hit a odd corner case that is a bit difficult to articulate for me that is largely dependent on emitting thin-link-bitcode.

This triggers to run ThinLTOBitcodeWriterPass which is not great and will cause the generated .o and .indexing.o (and potentially .bc if also emitting that) to have a different module hash since they are generated by other ThinLTOBitcodeWriterPass invocations. This can have a range if weird side affects (particularly with CFI enabled that sets EnableSplitLTOUnit). E.g. with CFI this strips debug info from .o and an upcoming LLVM change (merged in main llvm-project but not in the version used by Rust: aa91871), this causes a second ExternalLinkage global variable created in the .o that is not present in the .indexing.o file.

From tracing the code this seems to be the case because the .indexing.o is generated and emitted from optimize() (since #133250) but the generated .o is 'ignored' and then regenerated by codegen().

Now I'd love to get some input from experts on this part of the code on what the best way to solve this is.

I see two ways, 1) change the code back to emit .o, .indexing.o and .bc all at the same state from the same ThinLTOBitcodeWriterPass invocation. The change basically moving away from this suggests this is not intended or 2) Reuse the already generated .o file from running optimize() if it exists when running codegen() avoiding the second run of ThinLTOBitcodeWriterPass there. This looks like it's in line with how embedded bitcode is extracted from module.thin_lto_buffer.

cc'ing some folks who might be interested or touched this code before: @dianqk, @bjorn3, @rcvalle

Meta

rustc --version --verbose:

<version>

This was tested on commit 76c90957b7e422c4b9c45192b0197214d7de5a54.

The following test shows that ThinLTOBitcodeWriterPass runs twice since two different hashes are created:

tests/run-make/thin-link-bitcode/rmake.rs:

//@ ignore-backends: gcc

use run_make_support::{llvm_bcanalyzer, rustc};

fn main() {
    rustc()
        .arg("-Copt-level=2")
        .arg("-Ccodegen-units=1")
        .arg("-Cdebuginfo=2")
        .arg("-Zunstable-options")
        .crate_type("lib")
        .emit("llvm-bc,thin-link-bitcode")
        .input("program.rs")
        .run();
    check_thin_link_bitcode("program.bc", "program.indexing.o");
}

fn check_thin_link_bitcode(bc: &str, thin_link: &str) {
    let bc_dump = llvm_bcanalyzer()
        .arg("-dump")
        .input(bc)
        .run()
        .assert_stdout_contains("COMPILE_UNIT")
        .stdout_utf8();
    let thin_link_dump = llvm_bcanalyzer()
        .arg("-dump")
        .input(thin_link)
        .run()
        .assert_stdout_not_contains("COMPILE_UNIT")
        .stdout_utf8();

    let bc_hash = bc_dump.lines().find(|line| line.trim().starts_with("<HASH ")).unwrap();
    let thin_link_hash =
        thin_link_dump.lines().find(|line| line.trim().starts_with("<HASH ")).unwrap();
    assert_eq!(bc_hash, thin_link_hash);
}

tests/run-make/thin-link-bitcode/program.rs:

pub fn call_target(s: &[u32]) -> u32 {
    s.iter().sum()
}