Add bootstrap option to compile a tool with features
Add an option to specify which features to build a tool with, e.g. it will be useful to build Miri with tracing enabled:
```toml
tool-config.miri.features = ["tracing"]
```
See [this Zulip thread](https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Passing.20--features.20to.20Miri.20build.20using.20.2E.2Fx.2Epy/with/523564773) for the options considered. If the final decision will be different than what I wrote now, I will update the code as needed. The reason why the option is `tool-config.miri.features` instead of something like `tool-features.miri` is to possibly allow adding more tool-specific configurations in the future.
I didn't do any validation of the keys of the `tool-config` hashmap, since I saw that no validation is done on the `tools` hashset either.
I don't like much the fact that features can be chosen by various places of the codebase: `Step`s can have some fixed `extra_features`, `prepare_tool_cargo` will add features depending on some bootstrapping options, and the newly added option can also contribute features to tools. However I think it is out of scope of this PR to try to refactor all of that (if it even is refactorable), so I left a comment in the codebase explaining all of the sources of features I could find.
Assorted bootstrap cleanups (step 1)
Now that the stage0 redesign has landed, we can finally start cleaning up many things in bootstrap, and lord knows it deserves it! I plan to send many PRs once I figure out an incremental way forward, this is the first one of them. It doesn't actually change anything, just renames stuff and adds more documentation, but the rename is bitrotty, so I wanted to push the PR eagerly.
r? `@jieyouxu`
Add tracing import to execution context
In https://github.com/rust-lang/rust/pull/141909, we missed adding the trace_cmd import in the execution context module. This PR fixes that. Additionally, we are updating the mingw-check-2 check command to include BOOTSTRAP_TRACING=1 to help ensure we don't miss such cases in future PRs.
r? `@Kobzol`
Implement `//@ needs-target-std` compiletest directive
Closesrust-lang/rust#141863.
Needed to unblock rust-lang/rust#139244 and rust-lang/rust#141856.
### Summary
This PR implements a `//@ needs-target-std` compiletest directive that gates test execution based on whether the target supports std or not. For some cases, this should be preferred over e.g. some combination of `//@ ignore-none`, `//@ ignore-nvptx` and more[^none-limit].
### Implementation limitation
Unfortunately, since there is currently [no reliable way to determine from metadata whether a given target supports std or not](https://github.com/rust-lang/rust/issues/142296), we have to resort to a hack. Bootstrap currently determines whether or not a target supports std by a naive target tuple substring comparison: a target supports std if its target tuple does *not* contain one of `["-none", "nvptx", "switch"]` substrings. This PR simply pulls that hack out into `build_helpers` to avoid reimplementing the same hack in compiletest, and uses that logic to inform `//@ needs-target-std`.
### Auxiliary changes
This PR additionally changes a few run-make tests to use `//@ needs-target-std` over an inconsistent combination of target-based `ignore`s. This should help with rust-lang/rust#139244.
---
r? bootstrap
[^none-limit]: Notably, `target_os = "none"` is **not** a sufficient condition for "target does not support std"
Host is the machine where bootstrap runs, and this field represents the target of the (host) stage0/beta compiler. This is much clearer than `build`, which also conflicts with the `Build` struct, which is stored under the name `build` inside `Builder` (lol).
Add central execution context to bootstrap
This PR continues the effort toward command centralization as outlined in https://github.com/rust-lang/rust/issues/126819. It introduces a centralized execution context through which all commands will be executed. Previously, centralization was limited to build methods; this PR extends it to the `config` module and updates the remaining methods accordingly.
Best reviewed commit by commit.
r? ``@Kobzol``
To centralize this hack in one place with a backlink to the issue
tracking this hack, as this logic is also needed by compiletest to
implement a `//@ needs-target-std` directive.
add `Cargo.lock` to CI-rustc allowed list for non-CI env
Changes to dependencies usually require modifying `Cargo.toml`, which would already invalidate the CI-rustc cache if done in non-allowed paths. On non-CI environment, it should be safe to add `Cargo.lock` to the list of allowed paths as there is no real risk aside from a very rare false positive in cases like minor bumps to non-allowed path dependencies without modifying the `Cargo.toml` files.
Fixesrust-lang/rust#141986
Remap compiler vs non-compiler sources differently (bootstrap side)
See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083).
The path remapping and unremapping for compiler sources (distributed via `rustc-dev` dist component) is broken because bootstrap currently remaps all sources unconditionally (if remapping is enabled) to the `/rustc/{hash}` form. However, the `rustc-dev` dist component (compiler sources) and `rust-src` dist component (library sources) unpacks differently:
- `rust-src` unpacks sources to a path like `$sysroot/lib/rustlib/src/rust`, whereas
- `rustc-dev` unpacks sources to a path like `$sysroot/lib/rustlib/rustc-src/rust`[^note],
meaning that the compiler need to unremap them differently. But the same remapping means that the compiler has no way to distinguish between compiler and non-compiler (esp. standard library) sources. To remedy this, this PR adopts the approach of:
- remapping compiler sources (corresponding to `rustc-dev` dist component) with `/rustc-dev/{hash}` (this is `RemapScheme::Compiler`), and
- remapping non-compiler sources (corresponding to `rust-src` dist component or other non-compiler sources) with `/rustc/{hash}` (this is `RemapScheme::NonCompiler`).
A different remapping allows the compiler to reverse the remapping differently.
This PR implements the bootstrap side. A follow-up compiler-side change is needed to implement the unremapping change to address the reported issue completely.
This PR introduces another env var `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` that is made available to the compiler when building compiler sources to know what the remap scheme for `rustc-dev` (`RemapScheme::Compiler`) is. Compiler sources are built with the compiler remapping scheme.
As far as I know, this change should not introduce new regressions, because the compiler source unremapping (through `rustc-dev`) is already broken.
[^note]: (Notice the `src` vs `rustc-src` difference.)
bootstrap: build std sans leaf frame pointers
Sometimes leaf frame-pointers can impact LLVM inlining choices, and that can be a real problem for things like `mul_add`.
modularize the config module bootstrap
Currently, our `config` module is quite large over 3,000 lines, and handles a wide range of responsibilities. This PR aims to break it down into smaller, more focused submodules to improve readability and maintainability:
* **`toml`**: Introduces a dedicated `toml` submodule within the `config` module. Its sole purpose is to define configuration-related structs along with their corresponding deserialization logic. It also contains the `parse_inner` method, which serves as the central function for extracting relevant information from the TOML structs and constructing the final configuration.
* **`rust`, `dist`, `install`, `llvm`, `build`, `gcc`, and others**: Each of these modules contains TOML subsections specific to their domain, along with the logic necessary to convert them into parts of the final configuration struct.
* **`config/mod.rs`**: Contains shared types and enums used across multiple TOML subsections.
* **`config/config.rs`**: Houses the logic that integrates all the TOML subsections into the complete configuration struct.
r? `@kobzol`
