Remap compiler vs non-compiler sources differently (bootstrap side)
See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083).
The path remapping and unremapping for compiler sources (distributed via `rustc-dev` dist component) is broken because bootstrap currently remaps all sources unconditionally (if remapping is enabled) to the `/rustc/{hash}` form. However, the `rustc-dev` dist component (compiler sources) and `rust-src` dist component (library sources) unpacks differently:
- `rust-src` unpacks sources to a path like `$sysroot/lib/rustlib/src/rust`, whereas
- `rustc-dev` unpacks sources to a path like `$sysroot/lib/rustlib/rustc-src/rust`[^note],
meaning that the compiler need to unremap them differently. But the same remapping means that the compiler has no way to distinguish between compiler and non-compiler (esp. standard library) sources. To remedy this, this PR adopts the approach of:
- remapping compiler sources (corresponding to `rustc-dev` dist component) with `/rustc-dev/{hash}` (this is `RemapScheme::Compiler`), and
- remapping non-compiler sources (corresponding to `rust-src` dist component or other non-compiler sources) with `/rustc/{hash}` (this is `RemapScheme::NonCompiler`).
A different remapping allows the compiler to reverse the remapping differently.
This PR implements the bootstrap side. A follow-up compiler-side change is needed to implement the unremapping change to address the reported issue completely.
This PR introduces another env var `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` that is made available to the compiler when building compiler sources to know what the remap scheme for `rustc-dev` (`RemapScheme::Compiler`) is. Compiler sources are built with the compiler remapping scheme.
As far as I know, this change should not introduce new regressions, because the compiler source unremapping (through `rustc-dev`) is already broken.
[^note]: (Notice the `src` vs `rustc-src` difference.)
bootstrap: build std sans leaf frame pointers
Sometimes leaf frame-pointers can impact LLVM inlining choices, and that can be a real problem for things like `mul_add`.
modularize the config module bootstrap
Currently, our `config` module is quite large over 3,000 lines, and handles a wide range of responsibilities. This PR aims to break it down into smaller, more focused submodules to improve readability and maintainability:
* **`toml`**: Introduces a dedicated `toml` submodule within the `config` module. Its sole purpose is to define configuration-related structs along with their corresponding deserialization logic. It also contains the `parse_inner` method, which serves as the central function for extracting relevant information from the TOML structs and constructing the final configuration.
* **`rust`, `dist`, `install`, `llvm`, `build`, `gcc`, and others**: Each of these modules contains TOML subsections specific to their domain, along with the logic necessary to convert them into parts of the final configuration struct.
* **`config/mod.rs`**: Contains shared types and enums used across multiple TOML subsections.
* **`config/config.rs`**: Houses the logic that integrates all the TOML subsections into the complete configuration struct.
r? `@kobzol`
bootstrap: Fix file permissions when dereferencing symlinks
## Problem
When copying files in the bootstrap process with `dereference_symlinks = true`, we're incorrectly using the symlink's metadata to set permissions on the copied regular file, which results in the following error:
```
Warning: Failed to set file times for "/build/nix-build-rustc-1.86.0.drv-0/rustc-1.86.0-src/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-strip" (permissions: Permissions(FilePermissions { mode: 0o100000 (----------) })) error: Permission denied (os error 13)
```
Verbose Logs confirming the error:
```
TRACE: Found llvm-strip copy operation
Source: /n/nix/tech/store/n34yzv2n50p6lbjmx089vjym121wbl4j-llvm-19.1.7/bin/llvm-strip
Destination: /build/nix-build-rustc-1.86.0.drv-0/rustc-1.86.0-src/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-strip
Source is_symlink (via symlink_metadata): true
Source symlink_metadata permissions: 120000
Source symlink_metadata file_type: FileType { is_file: false, is_dir: false, is_symlink: true, .. }
Source is symlink pointing to: llvm-objcopy
Source raw mode: 120000
Source filetype: 120000
Setting permissions: Permissions(FilePermissions { mode: 0o120000 (l---------) })
Permission mode to set: 120000
Raw permission bits: 120000
File type bits being set: 120000
Permission bits being set: 0
WARNING: Attempting to set symlink file type (120000) on regular file!
WARNING: Setting zero permission bits will make file inaccessible!
Destination permissions after set_permissions: 100000
```
## Solution
After canonicalizing a symlink path, fetch the metadata of the target file instead of continuing to use the symlink's metadata. This ensures:
- Correct file type detection
- Proper permission bits for the target file
- Maintains existing behavior for non-symlink cases
## Testing
Verified fix resolves permission errors:
```
rustc> llvm-strip: Original metadata mode: 120000, is_symlink: true
rustc> llvm-strip: Target metadata mode after fix: 100555
rustc> llvm-strip: Final permissions mode: 100555
```
redesign stage 0 std follow-ups part2
Fixes three bugs:
1. `x check` fails when run on rustdoc without `download-rustc` enabled. (1st commit)
2. `x check` fails when run on the compiler with `download-rustc` enabled. (2nd commit)
3. `x test library` fails with `download-rustc` enabled. (3rd commit)
Fixesrust-lang/rust#142018 (case 1)
Fixes https://github.com/rust-lang/rust/issues/141983 (case 3)
In StdLink::run we subsequently recursively copy the initial sysroot
lib directory into the stage0-sysroot lib directory. If the initial
sysroot is a toolchain that includes the `rust-src` component (in
lib/rustlib/src/rust), if we add this symlink, that recursive copy
will overwrite the repo sources with the toolchain's sources.
Use ccache for stage0 tool builds
Now after the stage0 redesign, we can actually start ccaching the build of the compiler itself. We can also cache the bootstrap tools, since these are also built with the stage0 compiler.
Stage0 compiler builds are now being cached: https://github.com/rust-lang/rust/actions/runs/15397246267#summary-43321151192 (`..bootstrap::core::build_steps::compile::Rustc 483.10s 40.41s -91.6%`). It's not a gigantic win everywhere, but it should help. It seems to make the Linux jobs ~10 minute faster. It should be especially useful on PR builds after https://github.com/rust-lang/rust/pull/141948.
r? `@jieyouxu`
try-job: `x86_64-gnu-llvm-19*`
try-job: `x86_64-msvc*`
try-job: `x86_64-apple*`
try-job: `dist-x86_64-linux`
The module primarily defines the TOML representations of these subsections and their corresponding
configuration parsing implementations, which are ultimately integrated into the main configuration.
- `mod.rs` – Serves as the entry point for the TOML configuration and defines the `TomlConfig` struct along with its parsing implementation.
- `rust.rs` – Defines the `Rust` subsection struct and its configuration parsing implementation.
- `target.rs` – Defines the `Target` subsection struct and its configuration parsing implementation.
- `llvm.rs` – Defines the `Llvm` subsection struct and its configuration parsing implementation.
- `install.rs` – Defines the `Install` subsection struct and its configuration parsing implementation.
- `gcc.rs` – Defines the `Gcc` subsection struct and its configuration parsing implementation.
- `dist.rs` – Defines the `Dist` subsection struct and its configuration parsing implementation.
- `build.rs` – Defines the `Build` subsection struct and its configuration parsing implementation.
tools-aux ci runner: also cross-test doctests in Miri
Miri now supports running doctests across different targets. Let's use that to run the std doctests on aarch64-apple-darwin, i686-pc-windows-msvc.
try-job: x86_64-gnu-aux
Fix TLS model on bootstrap for cygwin
There aren't other targets that both use emutls and enable `has_thread_local`, so cygwin triggers this bug first.
r? mati865
See: https://github.com/rust-lang/rust/pull/141719#issuecomment-2925445263
``@jeremyd2019`` Could you check if this PR fixes the issue? I just found my pre-built stage-0 rustc was too old to build the current rustc :(
Exclude `CARGO_HOME` from `generate-copyright` in-tree determination
On Ferrocene, we noticed that in our releases the out-of-tree notices were not being included. When `x.py run generate-copyright` was ran on local development machines, it worked fine.
After some investigations ``@tshepang`` and I determined that the problem was that the cargo registry (located in `CARGO_HOME`) started with the source directory on CI jobs, and was being excluded by this line:
15825b7161/src/tools/generate-copyright/src/cargo_metadata.rs (L85-L88)
In Ferrocene's `run.sh` we set `CARGO_HOME` to be `build/cargo-home`: 96a45dd9a1/ferrocene/ci/run.sh (L34-L46) which caused this issue.
This PR passes the `CARGO_HOME` variable to the `generate-copyright` tool and expands the consideration of in-tree-ness to be aware of `CARGO_HOME`. It is an upstreaming of https://github.com/ferrocene/ferrocene/pull/1491.
## Testing
Run `CARGO_HOME=build/cargo-home ./x.py run generate-copyright` on `master`, then check `build/host/doc/COPYRIGHT` and look for out of tree dependencies (at the bottom).
Then, try running the same command in this branch.
