Don't panic if WASI_SDK_PATH not set when detecting compiler
The fedora packaging builds the wasm sysroot outside of the rust build system. Fedora applies a couple of patches related to wasm which I think make this possible. Not panicking seems consistent with the detection logic of other targets when they cannot find cc.
Drop `./x suggest`
This PR removes the current `./x suggest` implementation (rust-lang/rust#109933, rust-lang/rust#106249) and associated docs for several reasons:
1. Primarily, `./x suggest` is another "flow" in bootstrap that incurs extra complexity and more invariants that bootstrap has to maintain. This causes more friction when trying to investigate and fix staging problems. As far as I know, this flow has not been actively maintained in quite a while, and I'm not aware of interest in maintaining it. Bootstrap really could use less implementation complexity with a very limited maintenance bandwidth.
2. The current `./x suggest` implementation "bypasses" the usual stage defaults for the various check/build/test/etc. flows, and it's not really possible to have a stage default because `./x suggest --run` produces a *sequence* of suggestions like [`./x check`, `./x test library/std`, ..] and then tries to run all of them in sequence, based on which files are modified.
3. We've not seen a lot of interest both in using it or extending static/dynamic test suggestions. Last extensions were rust-lang/rust#117961 and rust-lang/rust#120763. I'm not convinced the extra implementation complexity is worth it. This was discussed in:
- [#t-infra/bootstrap > Dropping the current `./x suggest` flow implementation](https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Dropping.20the.20current.20.60.2E.2Fx.20suggest.60.20flow.20implementation/with/527456699)
- [#t-compiler > Dropping current `./x suggest` implementation](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Dropping.20current.20.60.2E.2Fx.20suggest.60.20implementation/with/527528696)
Closesrust-lang/rust#109933 (the current implementation is being removed).
Closesrust-lang/rust#143569 (by removing `./x suggest` altogether).
This is quite a bit of implementation complexity, yet it is quite
broken, and we don't have the maintenance bandwidth to address.
Remove the current implementation if only to reduce bootstrap's
implementation complexity; the `suggest` flow comes with its own set of
hacks.
Rename `stable_mir` and `rustc_smir`
This PR only renames the two crate names.
There is no doubt that we want to rename `stable_mir` to `rustc_public`, while it hasn't been discussed yet that what the new name for `rustc_smir` should be.
This PR proposes a new name for `rustc_smir`, that is `rustc_public_shim`, since `rustc_smir` now is mostly a proxy to do calls to rustc queries and the public API of rustc that is invoked by the `rustc_public` crate.
However, I don't think that name is good enough. I hope there would be a way better name.
r? `@oli-obk`
Run bootstrap tests sooner in the `x test` pipeline
With the recently added bootstrap snapshot tests, and in general with our plans to test more things in bootstrap, I feel like the original comment isn't accurate anymore. Recently, on several occasions I had to wait for 40+ minutes of CI just to find out that the bootstrap snapshot tests have failed. I think we should run bootstrap tests towards the beginning instead now.
r? ```@jieyouxu```
Change bootstrap's `tool.TOOL_NAME.features` to work on any subcommand
This is a followup to rust-lang/rust#142379 to make the bootstrap option `tool.TOOL_NAME.features` work on any subcommand instead of just build (so also run/test/...). I also made the `TOOL_NAME` comparisons look at the tool path instead of the tool name to determine to which tool a `TOOL_NAME` refers to, so you can specify tools by path like in other places of the bootstrap (e.g. `tool."tools/miri".features`).
bootstrap/miri: avoid rebuilds for test builds
When building Miri in its own repo, we always build with `--all-targets`:
a009612691/src/tools/miri/miri-script/src/util.rs (L167-L174)
This saves a bunch of time since some of Miri's dependencies get more features enabled by some of Miri's dev-dependencies, and they all get built twice otherwise if you do `cargo build && cargo test` (which is typically what you end up doing inside `./miri test` and also inside `./x test miri`).
This applies the same approach to bootstrap, drastically reducing the edit-compile cycle for Miri work here. :)
Fix command trace
With the recent developments in centralization of command execution, we somehow broke the traces for command execution. This PR fixes that and add trace to stream command execution as well.
r? ````@Kobzol````
clippy: fix test filtering when TESTNAME is empty
Fixes https://github.com/rust-lang/rust/issues/143824. Turns out bootstrap was just fine, the TESTNAME logic in clippy was wrong... I still made this a rustc PR as that's where I did all the debugging.
The bootstrap change is not really related, but it's comment-only so not worth a separate PR... adding the `test_args` is also part of what `prepare_cargo_test` would usually do so let's group the code properly.
Add --compile-time-deps argument for x check
Together with skipping building C++ code in rustc_llvm for check, this reduces the amount of time it takes to do the x check for rust-analyzer analysis from 12m16s to 3m06s when the bootstrap compiler is already downloaded.
Access `wasi_sdk_path` instead of reading environment variable in bootstrap
Small cleanup to remove an environment variable read that we have performed earlier in bootstrap already.
Remove format short command trait
Since we no longer have traces of the vanilla command, and we're already implementing format_short_command for CommandFingerprint, we can use it directly from the fingerprint. This PR removes the standalone format_short_command trait and moves its implementation under CommandFingerprint.
This reduces the amount of time it takes to do the x check for
rust-analyzer analysis from 12m16s to 3m34s when the bootstrap compiler
is already downloaded.
Fix `--skip-std-check-if-no-download-rustc`
Since https://github.com/rust-lang/rust/pull/143048, we now explicitly set the build compiler for `check::Std`, which caused it to be built before we checked `--skip-std-check-if-no-download-rustc`. So I moved the check earlier to `make_run`, which resolves it.
I also added a regression test for this. Sadly we can't really test for the positive case easily (when download-ci-rustc is enabled), but we can test the negative cases, where it is disabled.
Fixes: https://github.com/rust-lang/rust/issues/143705
r? ```@RalfJung```
Rollup of 9 pull requests
Successful merges:
- rust-lang/rust#143446 (use `--dynamic-list` for exporting executable symbols)
- rust-lang/rust#143590 (Fix weird rustdoc output when single and glob reexport conflict on a name)
- rust-lang/rust#143599 (emit `.att_syntax` when global/naked asm use that option)
- rust-lang/rust#143615 (Fix handling of no_std targets in `doc::Std` step)
- rust-lang/rust#143632 (fix: correct parameter names in LLVMRustBuildMinNum and LLVMRustBuildMaxNum FFI declarations)
- rust-lang/rust#143640 (Constify `Fn*` traits)
- rust-lang/rust#143651 (Win: Use exceptions with empty data for SEH panic exception copies instead of a new panic)
- rust-lang/rust#143660 (Disable docs for `compiler-builtins` and `sysroot`)
- rust-lang/rust#143665 ([rustdoc-json] Add tests for `#[doc(hidden)]` handling of items.)
r? `@ghost`
`@rustbot` modify labels: rollup
Disable docs for `compiler-builtins` and `sysroot`
Bootstrap already had a manual doc filter for the `sysroot` crate, but
other library crates keep themselves out of the public docs by setting
`[lib] doc = false` in their manifest. This seems like a better solution
to hide `compiler-builtins` docs, and removes the `sysroot` hack too.
Fixesrust-lang/rust#143215 (after backport)
```@rustbot``` label beta-nominated
Fix handling of no_std targets in `doc::Std` step
The previous logic was wrong for no_std targets, it just didn't do anything. The logic was added there because by default, the `Std` step would otherwise have a list of all std crates to check, but these would fail for no_std targets. What has to happen instead is to select the default set of packages to check/doc/build, which currently happens in the `std_cargo` function, but the `self.crates` list was overriding that.
In general, using `crates: Vec<String>` in the `Std` steps is quite fishy, because it's difficult to distinguish between all crates (either they are all enumerated or `crates` is empty) and the default (e.g. `x <kind> [library]`) vs a subset (e.g. `x <kind> core`). I wanted to improve that using an enum that would distinguish these situations, avoid passing `-p` for all of the crates explicitly, and unify the selection of packages to compile/check/... in `std_cargo`, based on this enum.
However, I found out from some other bootstrap comments that when you pass `-p` explicitly for all crates, cargo behaves differently (apparently for check it will also check targets/examples etc. with `-p`, but not without it). Furthermore, the doc step has a special case where it does not document the `sysroot` package. So as usually, unifying this logic would get into some edge cases... So instead I opted for a seemingly simpler solution, where I try to prefilter only two allowed crates (core and alloc) for no_std targets in the `std_crates_for_run_make` function.
It's not perfect, but I think it's better than the status quo (words to live by when working on bootstrap...).
Fixes [this Zulip topic](https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/docs.20for.20non-host.20targets.3F).
r? `@jieyouxu`
Add profiler to bootstrap command
This PR adds command profiling to the bootstrap command. It tracks the total execution time and records cache hits for each command. It also provides the ability to export execution result to a JSON file. Integrating this with Chrome tracing could further enhance observability.
r? `@Kobzol`
tidy: add support for `--extra-checks=auto:` feature
in preparation for rust-lang/rust#142924
also heavily refactored the parsing of the `--extra-checks` argument to warn about improper usage.
cc ```@GuillaumeGomez```
r? ```@Kobzol```
Since https://github.com/rust-lang/rust/pull/143048, we now explicitly set the build compiler for `check::Std`, which caused it to be built before we checked `--skip-std-check-if-no-download-rustc`. So I moved the check earlier to `make_run`, which resolves it.
I also added a regression test for this. Sadly we can't really test for the positive case easily (when download-ci-rustc is enabled), but we can test the negative cases, where it is disabled.
