Add `ToolTarget` to bootstrap
Oh, you thought I'm done with refactoring bootstrap tools? Na-ah, think again! After the failure of https://github.com/rust-lang/rust/pull/143581, `ToolTarget` is back with a vengeance. This time, I implemented the test changes and tool cleanups without forcing these tools to be built with the stage0 compiler.
There are still some small wins though, `LlvmBitcodeLinker` now starts at stage 1, and not stage 2. Cargo should also be ported to this new mode, but I'm leaving that for a follow-up PR.
Hopefully X-th time's the charm 🤞
r? `@jieyouxu`
bootstrap: Detect musl hosts
Currently, all non-Android Linux hosts are assumed to be using glibc. This obviously isn't very portable and will currently result in downloading a stage0 toolchain for glibc even on musl hosts.
There are multiple ways to detect musl somewhat reliably, but the easiest option is to check for the python target, which is exposed in `sys.implementation._multiarch` and has values like "x86_64-linux-gnu" or "powerpc64le-linux-musl".
Currently, all non-Android Linux hosts are assumed to be using glibc.
This obviously isn't very portable and will currently result in
downloading a stage0 toolchain for glibc even on musl hosts.
There are multiple ways to detect musl somewhat reliably, but the
easiest option is to check for the python SOABI config variable, which
has values like "cpython-313-x86_64-linux-gnu" or
"cpython-313-powerpc64-linux-musl".
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
Copy GCC sources into the build directory even outside CI
It takes ~3.5s on my Linux notebook to perform the copy, but it should only be executed when we actually go build GCC, and that will almost certainly take much longer :) So I think it should be fine. At least we won't be polluting the source directory for local builds.
Fixes: https://github.com/rust-lang/rust/issues/143986
r? `````@nikic`````
Boostrap: add warning on `optimize = false`
I recently came across a bug that can be traced back to the use of `optimize = false` in `bootstrap.toml` in combination with other settings. Following [this](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Missing.20box-related.20symbols.20with.20panic.20.3D.20'abort'/with/528992909) conversation, this PR adds a warning from `bootstrap` when `optimize = false` is used.
I notice that in the same file I edited there are two different styles for warnings (`WARN`, `Warning`). I used `WARNING` because, by happenstance, when testing I got a `WARNING` that I didn't set a change id: let me know if I can unify the styles in the file I edited.
Don't panic if WASI_SDK_PATH not set when detecting compiler
The fedora packaging builds the wasm sysroot outside of the rust build system. Fedora applies a couple of patches related to wasm which I think make this possible. Not panicking seems consistent with the detection logic of other targets when they cannot find cc.
Drop `./x suggest`
This PR removes the current `./x suggest` implementation (rust-lang/rust#109933, rust-lang/rust#106249) and associated docs for several reasons:
1. Primarily, `./x suggest` is another "flow" in bootstrap that incurs extra complexity and more invariants that bootstrap has to maintain. This causes more friction when trying to investigate and fix staging problems. As far as I know, this flow has not been actively maintained in quite a while, and I'm not aware of interest in maintaining it. Bootstrap really could use less implementation complexity with a very limited maintenance bandwidth.
2. The current `./x suggest` implementation "bypasses" the usual stage defaults for the various check/build/test/etc. flows, and it's not really possible to have a stage default because `./x suggest --run` produces a *sequence* of suggestions like [`./x check`, `./x test library/std`, ..] and then tries to run all of them in sequence, based on which files are modified.
3. We've not seen a lot of interest both in using it or extending static/dynamic test suggestions. Last extensions were rust-lang/rust#117961 and rust-lang/rust#120763. I'm not convinced the extra implementation complexity is worth it. This was discussed in:
- [#t-infra/bootstrap > Dropping the current `./x suggest` flow implementation](https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Dropping.20the.20current.20.60.2E.2Fx.20suggest.60.20flow.20implementation/with/527456699)
- [#t-compiler > Dropping current `./x suggest` implementation](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Dropping.20current.20.60.2E.2Fx.20suggest.60.20implementation/with/527528696)
Closesrust-lang/rust#109933 (the current implementation is being removed).
Closesrust-lang/rust#143569 (by removing `./x suggest` altogether).
This is quite a bit of implementation complexity, yet it is quite
broken, and we don't have the maintenance bandwidth to address.
Remove the current implementation if only to reduce bootstrap's
implementation complexity; the `suggest` flow comes with its own set of
hacks.
Rename `stable_mir` and `rustc_smir`
This PR only renames the two crate names.
There is no doubt that we want to rename `stable_mir` to `rustc_public`, while it hasn't been discussed yet that what the new name for `rustc_smir` should be.
This PR proposes a new name for `rustc_smir`, that is `rustc_public_shim`, since `rustc_smir` now is mostly a proxy to do calls to rustc queries and the public API of rustc that is invoked by the `rustc_public` crate.
However, I don't think that name is good enough. I hope there would be a way better name.
r? `@oli-obk`
Run bootstrap tests sooner in the `x test` pipeline
With the recently added bootstrap snapshot tests, and in general with our plans to test more things in bootstrap, I feel like the original comment isn't accurate anymore. Recently, on several occasions I had to wait for 40+ minutes of CI just to find out that the bootstrap snapshot tests have failed. I think we should run bootstrap tests towards the beginning instead now.
r? ```@jieyouxu```
Change bootstrap's `tool.TOOL_NAME.features` to work on any subcommand
This is a followup to rust-lang/rust#142379 to make the bootstrap option `tool.TOOL_NAME.features` work on any subcommand instead of just build (so also run/test/...). I also made the `TOOL_NAME` comparisons look at the tool path instead of the tool name to determine to which tool a `TOOL_NAME` refers to, so you can specify tools by path like in other places of the bootstrap (e.g. `tool."tools/miri".features`).
bootstrap/miri: avoid rebuilds for test builds
When building Miri in its own repo, we always build with `--all-targets`:
a009612691/src/tools/miri/miri-script/src/util.rs (L167-L174)
This saves a bunch of time since some of Miri's dependencies get more features enabled by some of Miri's dev-dependencies, and they all get built twice otherwise if you do `cargo build && cargo test` (which is typically what you end up doing inside `./miri test` and also inside `./x test miri`).
This applies the same approach to bootstrap, drastically reducing the edit-compile cycle for Miri work here. :)
Fix command trace
With the recent developments in centralization of command execution, we somehow broke the traces for command execution. This PR fixes that and add trace to stream command execution as well.
r? ````@Kobzol````
clippy: fix test filtering when TESTNAME is empty
Fixes https://github.com/rust-lang/rust/issues/143824. Turns out bootstrap was just fine, the TESTNAME logic in clippy was wrong... I still made this a rustc PR as that's where I did all the debugging.
The bootstrap change is not really related, but it's comment-only so not worth a separate PR... adding the `test_args` is also part of what `prepare_cargo_test` would usually do so let's group the code properly.
Add --compile-time-deps argument for x check
Together with skipping building C++ code in rustc_llvm for check, this reduces the amount of time it takes to do the x check for rust-analyzer analysis from 12m16s to 3m06s when the bootstrap compiler is already downloaded.
Access `wasi_sdk_path` instead of reading environment variable in bootstrap
Small cleanup to remove an environment variable read that we have performed earlier in bootstrap already.
Remove format short command trait
Since we no longer have traces of the vanilla command, and we're already implementing format_short_command for CommandFingerprint, we can use it directly from the fingerprint. This PR removes the standalone format_short_command trait and moves its implementation under CommandFingerprint.
This reduces the amount of time it takes to do the x check for
rust-analyzer analysis from 12m16s to 3m34s when the bootstrap compiler
is already downloaded.
Fix `--skip-std-check-if-no-download-rustc`
Since https://github.com/rust-lang/rust/pull/143048, we now explicitly set the build compiler for `check::Std`, which caused it to be built before we checked `--skip-std-check-if-no-download-rustc`. So I moved the check earlier to `make_run`, which resolves it.
I also added a regression test for this. Sadly we can't really test for the positive case easily (when download-ci-rustc is enabled), but we can test the negative cases, where it is disabled.
Fixes: https://github.com/rust-lang/rust/issues/143705
r? ```@RalfJung```
Rollup of 9 pull requests
Successful merges:
- rust-lang/rust#143446 (use `--dynamic-list` for exporting executable symbols)
- rust-lang/rust#143590 (Fix weird rustdoc output when single and glob reexport conflict on a name)
- rust-lang/rust#143599 (emit `.att_syntax` when global/naked asm use that option)
- rust-lang/rust#143615 (Fix handling of no_std targets in `doc::Std` step)
- rust-lang/rust#143632 (fix: correct parameter names in LLVMRustBuildMinNum and LLVMRustBuildMaxNum FFI declarations)
- rust-lang/rust#143640 (Constify `Fn*` traits)
- rust-lang/rust#143651 (Win: Use exceptions with empty data for SEH panic exception copies instead of a new panic)
- rust-lang/rust#143660 (Disable docs for `compiler-builtins` and `sysroot`)
- rust-lang/rust#143665 ([rustdoc-json] Add tests for `#[doc(hidden)]` handling of items.)
r? `@ghost`
`@rustbot` modify labels: rollup
Disable docs for `compiler-builtins` and `sysroot`
Bootstrap already had a manual doc filter for the `sysroot` crate, but
other library crates keep themselves out of the public docs by setting
`[lib] doc = false` in their manifest. This seems like a better solution
to hide `compiler-builtins` docs, and removes the `sysroot` hack too.
Fixesrust-lang/rust#143215 (after backport)
```@rustbot``` label beta-nominated
