rust/compiler
bors e1b06f7730 Auto merge of #139453 - compiler-errors:incr, r=jieyouxu
Prepend temp files with per-invocation random string to avoid temp filename conflicts

https://github.com/rust-lang/rust/issues/139407 uncovered a very subtle unsoundness with incremental codegen, failing compilation sessions (due to assembler errors), and the "prefer hard linking over copying files" strategy we use in the compiler for file management.

Specifically, imagine we're building a single file 3 times, all with `-Csave-temps -Cincremental=...`. Let's call the object file we're building for the codegen unit for `main` "`XXX.o`" just for clarity since it's probably some gigantic hash name:

```
#[inline(never)]
#[cfg(any(rpass1, rpass3))]
fn a() -> i32 {
    0
}

#[cfg(any(cfail2))]
fn a() -> i32 {
    1
}

fn main() {
    evil::evil();
    assert_eq!(a(), 0);
}

mod evil {
    #[cfg(any(rpass1, rpass3))]
    pub fn evil() {
        unsafe {
            std::arch::asm!("/*  */");
        }
    }

    #[cfg(any(cfail2))]
    pub fn evil() {
        unsafe {
            std::arch::asm!("missing");
        }
    }
}
```

Session 1 (`rpass1`):
* Type-check, borrow-check, etc.
* Serialize the dep graph to the incremental working directory `.../s-...-working/`.
* Codegen object file to a temp file `XXX.rcgu.o` which is spit out in the cwd.
* Hard-link[^1] `XXX.rcgu.o` to the incremental working directory `.../s-...-working/XXX.o`.
* Save-temps option means we don't delete `XXX.rgcu.o`.
* Link the binary and stuff.
* Finalize[^2] the working incremental session by renaming `.../s-...-working` to ` s-...-asjkdhsjakd` (some other finalized incr comp session dir name).

Session 2 (`cfail2`):
* Load artifacts from the previous *finalized* incremental session, namely the dep graph.
* Type-check, borrow-check, etc. since the file has changed, so most dep graph nodes are red.
* Serialize the dep graph to the incremental working directory `.../s-...-working/`.
* Codegen object file to a temp file `XXX.rcgu.o`. **HERE IS THE PROBLEM**: The hard-link is still set up to point to the inode from `XXX.o` from the first session, so this also modifies the `XXX.o` in the previous finalized session directory.
* Codegen emits an error b/c `missing` is not an instruction, so we abort before finalizing the incremental session. Specifically, this means that the *previous* session is the last finalized session.

Session 3 (`rpass3`):
* Load artifacts from the previous *finalized* incremental session, namely the dep graph. NOTE that this is from session 1.
* All the dep graph nodes are green since we are basically replaying session 1.
* codegen object file `XXX.o`, which is detected as *reused* from session 1 since dep nodes were green. That means we **reuse** `XXX.o` which had been dirtied from session 2.
* Link the binary and stuff.

This results in a binary which reuses some of the build artifacts from session 2, but thinks it's from session 1.

At this point, I hope it's clear to see that the incremental results from session 1 were dirtied from session 2, but we reuse them as if session 1 was the previous (finalized) incremental session we ran. This is at best really buggy, and at worst **unsound**.

This isn't limited to `-C save-temps`, since there are other combinations of flags that may keep around temporary files (hard linked) in the working directory (like `-C debuginfo=1 -C split-debuginfo=unpacked` on darwin, for example).

---

This PR implements a fix which is to prepend temp filenames with a random string that is generated per invocation of rustc. This string is not *deterministic*, but temporary files are transient anyways, so I don't believe this is a problem.

That means that temp files are now something like... `{crate-name}.{cgu}.{invocation_temp}.rcgu.o`, where `{invocation_temp}` is the new temporary string we generate per invocation of rustc.

Fixes https://github.com/rust-lang/rust/issues/139407

[^1]: 175dcc7773/compiler/rustc_fs_util/src/lib.rs (L60)
[^2]: 175dcc7773/compiler/rustc_incremental/src/persist/fs.rs (L1-L40)
2025-04-11 13:59:33 +00:00
..
rustc Revert "Use workspace lints for crates in compiler/ #138084" 2025-03-10 18:12:47 +08:00
rustc_abi Remove redundant words 2025-04-09 18:46:50 +08:00
rustc_arena Remove #![warn(unreachable_pub)] from all compiler/ crates. 2025-03-11 13:14:21 +11:00
rustc_ast Rename some name variables as ident. 2025-04-10 09:30:55 +10:00
rustc_ast_ir Use -Wunused_crate_dependencies for compiler crates. 2025-03-20 08:59:43 +11:00
rustc_ast_lowering Rollup merge of #139641 - BoxyUwU:allow_parend_array_len_infer, r=compiler-errors 2025-04-11 13:31:49 +10:00
rustc_ast_passes Rollup merge of #139035 - nnethercote:PatKind-Missing, r=oli-obk 2025-04-07 22:29:17 +10:00
rustc_ast_pretty Rename some name variables as ident. 2025-04-10 09:30:55 +10:00
rustc_attr_data_structures add rustc_macro_edition_2021 2025-03-19 17:37:35 +01:00
rustc_attr_parsing Avoid kw::Empty when dealing with rustc_allowed_through_unstable_modules. 2025-03-25 16:48:03 +11:00
rustc_baked_icu_data Add unreachable_pub to RUSTC_LINT_FLAGS for compiler/ crates. 2025-03-11 13:14:21 +11:00
rustc_borrowck Rollup merge of #139510 - nnethercote:name-to-ident, r=fee1-dead 2025-04-10 17:27:14 +02:00
rustc_builtin_macros Rollup merge of #139510 - nnethercote:name-to-ident, r=fee1-dead 2025-04-10 17:27:14 +02:00
rustc_codegen_cranelift Auto merge of #139453 - compiler-errors:incr, r=jieyouxu 2025-04-11 13:59:33 +00:00
rustc_codegen_gcc Prepend temp files with a string per invocation of rustc 2025-04-07 20:48:40 +00:00
rustc_codegen_llvm Auto merge of #139453 - compiler-errors:incr, r=jieyouxu 2025-04-11 13:59:33 +00:00
rustc_codegen_ssa Auto merge of #139453 - compiler-errors:incr, r=jieyouxu 2025-04-11 13:59:33 +00:00
rustc_const_eval Make LevelAndSource a struct 2025-04-03 09:17:55 +00:00
rustc_data_structures Auto merge of #139011 - Zoxc:no-rayon-iters, r=oli-obk 2025-04-11 07:34:27 +00:00
rustc_driver Revert "Use workspace lints for crates in compiler/ #138084" 2025-03-10 18:12:47 +08:00
rustc_driver_impl Rollup merge of #138682 - Alexendoo:extra-symbols, r=fee1-dead 2025-04-11 13:31:44 +10:00
rustc_error_codes Remove some dead or leftover code related to rustc-intrinsic abi removal 2025-04-09 07:57:13 +00:00
rustc_error_messages update cfgs 2025-04-09 12:29:59 +01:00
rustc_errors Rollup merge of #139468 - compiler-errors:has_stashed_diagnostic, r=oli-obk 2025-04-09 14:52:38 +02:00
rustc_expand Address review comments. 2025-04-10 09:39:21 +10:00
rustc_feature replace version placeholder 2025-04-09 12:29:59 +01:00
rustc_fluent_macro Remove #![warn(unreachable_pub)] from all compiler/ crates. 2025-03-11 13:14:21 +11:00
rustc_fs_util Revert "Use workspace lints for crates in compiler/ #138084" 2025-03-10 18:12:47 +08:00
rustc_graphviz Remove #![warn(unreachable_pub)] from all compiler/ crates. 2025-03-11 13:14:21 +11:00
rustc_hashes Revert "Use workspace lints for crates in compiler/ #138084" 2025-03-10 18:12:47 +08:00
rustc_hir Rollup merge of #138682 - Alexendoo:extra-symbols, r=fee1-dead 2025-04-11 13:31:44 +10:00
rustc_hir_analysis Rollup merge of #137447 - folkertdev:simd-extract-insert-dyn, r=scottmcm 2025-04-11 13:31:43 +10:00
rustc_hir_pretty Rename some name variables as ident. 2025-04-10 09:30:55 +10:00
rustc_hir_typeck Rollup merge of #139510 - nnethercote:name-to-ident, r=fee1-dead 2025-04-10 17:27:14 +02:00
rustc_incremental Auto merge of #138629 - Zoxc:graph-anon-hashmap, r=oli-obk 2025-03-24 15:02:09 +00:00
rustc_index Use {Decodable,Encodable}_NoContext in type_ir 2025-03-15 06:34:36 +00:00
rustc_index_macros Add unreachable_pub to RUSTC_LINT_FLAGS for compiler/ crates. 2025-03-11 13:14:21 +11:00
rustc_infer clean code: remove Deref<Target=RegionKind> impl for Region and use .kind() 2025-04-08 10:51:41 +08:00
rustc_interface Auto merge of #139453 - compiler-errors:incr, r=jieyouxu 2025-04-11 13:59:33 +00:00
rustc_lexer Replace rustc_lexer/unescape with rustc-literal-escaper crate 2025-04-04 14:44:45 +02:00
rustc_lint Rollup merge of #139510 - nnethercote:name-to-ident, r=fee1-dead 2025-04-10 17:27:14 +02:00
rustc_lint_defs impl !PartialOrd for HirId 2025-04-03 09:22:21 +00:00
rustc_llvm Update the minimum external LLVM to 19 2025-04-05 11:44:38 -07:00
rustc_log Use -Wunused_crate_dependencies for compiler crates. 2025-03-20 08:59:43 +11:00
rustc_macros Allow drivers to supply a list of extra symbols to intern 2025-04-10 13:39:23 +00:00
rustc_metadata Auto merge of #139011 - Zoxc:no-rayon-iters, r=oli-obk 2025-04-11 07:34:27 +00:00
rustc_middle Auto merge of #139453 - compiler-errors:incr, r=jieyouxu 2025-04-11 13:59:33 +00:00
rustc_mir_build Make use generated TerminatorKind::Call have call_source Use 2025-04-07 16:53:11 -03:00
rustc_mir_dataflow Auto merge of #138414 - matthiaskrgr:rollup-9ablqdb, r=matthiaskrgr 2025-03-12 17:27:43 +00:00
rustc_mir_transform Auto merge of #139327 - cjgillot:gvn-place, r=oli-obk 2025-04-09 08:50:33 +00:00
rustc_monomorphize Auto merge of #139011 - Zoxc:no-rayon-iters, r=oli-obk 2025-04-11 07:34:27 +00:00
rustc_next_trait_solver Auto merge of #139000 - compiler-errors:rigid-missing-item, r=lcnr 2025-04-10 04:03:59 +00:00
rustc_parse Avoid an empty trait name in impl blocks. 2025-04-09 15:01:14 +10:00
rustc_parse_format Update rustc-literal-escaper version to 0.0.2 2025-04-04 22:26:10 +02:00
rustc_passes Rollup merge of #139455 - Skgland:remove_rust-intrinsic_ABI, r=oli-obk 2025-04-07 22:29:20 +10:00
rustc_pattern_analysis Fix stack overflow in exhaustiveness due to recursive HIR opaque type values 2025-04-07 22:41:48 +00:00
rustc_privacy privacy: Visit types and traits in impls in type privacy lints 2025-03-25 12:40:02 +03:00
rustc_query_impl Add a dep kind for use of the anon node with zero dependencies 2025-04-02 07:35:05 +02:00
rustc_query_system Reuse the index from promoted nodes when coloring executed tasks 2025-04-05 14:41:08 +02:00
rustc_resolve Rollup merge of #139510 - nnethercote:name-to-ident, r=fee1-dead 2025-04-10 17:27:14 +02:00
rustc_sanitizers Encode synthetic by-move coroutine body with a different DefPathData 2025-03-30 22:53:21 +00:00
rustc_serialize Convert rustc_serialize integration tests to unit tests. 2025-03-20 08:59:50 +11:00
rustc_session Prepend temp files with a string per invocation of rustc 2025-04-07 20:48:40 +00:00
rustc_smir remove compiler support for extern "rust-intrinsic" blocks 2025-04-06 21:32:58 +02:00
rustc_span Rollup merge of #138682 - Alexendoo:extra-symbols, r=fee1-dead 2025-04-11 13:31:44 +10:00
rustc_symbol_mangling clean code: remove Deref<Target=RegionKind> impl for Region and use .kind() 2025-04-08 10:51:41 +08:00
rustc_target remove compiler support for extern "rust-intrinsic" blocks 2025-04-06 21:32:58 +02:00
rustc_trait_selection Rollup merge of #139564 - compiler-errors:deeply-norm, r=lcnr 2025-04-11 13:31:48 +10:00
rustc_traits Rollup merge of #138394 - lcnr:yeet-variant, r=compiler-errors 2025-03-12 10:19:32 -07:00
rustc_transmute Add #[cfg(test)] for Transition in dfa 2025-03-18 07:17:16 +00:00
rustc_ty_utils Use a query rather than recomputing the tail repeatedly 2025-04-09 20:26:58 +00:00
rustc_type_ir Auto merge of #139000 - compiler-errors:rigid-missing-item, r=lcnr 2025-04-10 04:03:59 +00:00
rustc_type_ir_macros Fold visit into ty 2025-03-15 06:34:36 +00:00
stable_mir let rustc_smir host stable_mir for refactoring 2025-04-05 18:23:07 +08:00