Switch some more Steps to `builder.msg`
In particular, this hides compiletest suites under a dropdown - some builders are configured with `verbose-tests`, which makes the suites take up many thousands of lines.
before:
```
Documenting stage1 cargo (x86_64-unknown-linux-gnu)
Check compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
```
after:
```
::group::Documenting stage1 cargo (x86_64-unknown-linux-gnu)
::group::Testing stage1 compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu)
```
found this from https://github.com/rust-lang-ci/rust/actions/runs/5366993848/jobs/9736792611
r? `@oli-obk`
Test the cargo args generated by bootstrap.py
I recommend reviewing this commit-by-commit using the instructions in https://rustc-dev-guide.rust-lang.org/git.html#moving-large-sections-of-code.
- Test cargo arguments passed by bootstrap.py
This moves a lot of code around, but the logic itself is not too terribly complicated.
- Move almost all logic in `def bootstrap` to the `RustBuild` class, to avoid mixing setting configuration with running commands
- Update various doctests to the new (more complete) RustBuild config. In particular, don't pretend that `bin_root` supports `build` being unset.
- Change `parse_args` not to use a global, to allow testing it
- Set BUILD_DIR appropriately so bootstrap.py doesn't panic because cargo isn't found
- Allow passing arguments to `bootstrap_test.py`
Previous, it used the built-in test runner, which doesn't support options unless they're manually passed in the script.
- Fix progress messages for configure in bootstrap_test.py
Before it would unconditionally print `configure-args = []`.
r? `@albertlarsan68` cc https://github.com/rust-lang/rust/pull/112089https://github.com/rust-lang/rust/pull/111979#issuecomment-1568525699
In particular, this hides compiletest suites under a dropdown - some builders are configured with `verbose-tests`, which makes the suites take up many thousands of lines.
before:
```
Documenting stage1 cargo (x86_64-unknown-linux-gnu)
Check compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
```
after:
```
::group::Documenting stage1 cargo (x86_64-unknown-linux-gnu)
::group::Testing stage1 compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu)
```
The old code actually replaced `.json` with `.bak` (so, `settings.bak`),
rather than appending `.bak` as claimed (`settings.json.bak`).
`Path::set_extension` can instead be used with dots:
> The new extension may contain dots and will be used in its entirety,
> but only the part after the final dot will be reflected in
> self.extension.
This moves a lot of code around, but the logic itself is not too terribly complicated.
- Move almost all logic in `def bootstrap` to the `RustBuild` class, to avoid mixing setting configuration with running commands
- Update various doctests to the new (more complete) RustBuild config. In particular, don't pretend that `bin_root` supports `build` being unset.
- Change `parse_args` not to use a global, to allow testing it
- Set BUILD_DIR appropriately so bootstrap.py doesn't panic because cargo isn't found
Don't try to auto-bless 32-bit `mir-opt` tests on ARM Mac hosts
#112418 added special support for automatically blessing 32-bit output on 64-bit hosts, for the subset of `mir-opt` tests that are pointer-width-dependent.
This relies on the 64-bit host having some corresponding 32-bit target that can be built “easily”. For most 64-bit hosts this is fine, but ARM Macs don't have a corresponding 32-bit target. (There have never been 32-bit ARM Macs, and ARM Macs don't have the libraries needed for building `i686-apple-darwin`.)
There is an entry for `("i686-apple-darwin", "aarch64-apple-darwin")` in the list of corresponding 32-bit platforms, but this doesn't actually work on ARM Macs. Instead, the bootstrap invocation fails to build the necessary 32-bit target support, and nothing gets tested or blessed.
According to [this Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Can't.20bless.20any.20mir-opt.20tests.20on.20aarch64.20Mac/near/367109789), that mapping was only added because the author assumed it would work. But since it doesn't actually work on ARM Macs, the solution is to just remove that mapping.
With the mapping removed, ARM Macs still can't auto-bless 32-bit output (they will see a warning instead), but at least they can now bless the output of `mir-opt` tests that don't care about pointer width.
Blessing 32-bit tests on 64-bit hosts relies on having a corresponding 32-bit
target that can be built "easily" on those hosts.
ARM Macs don't have a corresponding 32-bit target, so trying to build one is
usually going to fail.
Add support for targets without unwinding in `mir-opt`, and improve `--bless` for it
The main goal of this PR is to add support for targets without unwinding support in the `mir-opt` test suite, by adding the `EMIT_MIR_FOR_EACH_PANIC_STRATEGY` comment. Similarly to 32bit vs 64bit, when that comment is present, blessed output files will have the `.panic-unwind` or `.panic-abort` suffix, and the right one will be chosen depending on the target's panic strategy.
The `EMIT_MIR_FOR_EACH_PANIC_STRATEGY` comment replaced all the `ignore-wasm32` comments in the `mir-opt` test suite, as those comments were added due to `wasm32` being a target without unwinding support. The comment was also added on other tests that were only executed on x86 but were still panic strategy dependent.
The `mir-opt` suite was then blessed, which caused a ton of churn as most of the existing output files had to be renamed and (mostly) duplicated with the abort strategy.
---
After [asking on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/mir-opt.20tests.20and.20panic.3Dabort), the main concern about this change is it'd make blessing the `mir-opt` suite even harder, as you'd need to both bless it with an unwinding target and an aborting target. This exacerbated the current situation, where you'd need to bless it with a 32bit and a 64bit target already.
Because of that, this PR also makes significant enhancements to `--bless` for the `mir-opt` suite, where it will automatically bless the suite four times with different targets, while requiring minimal cross-compilation.
To handle the 32bit vs 64bit blessing, there is now an hardcoded list of target mapping between 32bit and 64bit. The goal of the list is to find a related target that will *probably* work without requiring additional cross-compilation toolchains on the system. If a mapping is found, bootstrap will bless the suite with both targets, otherwise just with the current target.
To handle the panic strategy blessing (abort vs unwind), I had to resort to what I call "synthetic targets". For each of the target we're blessing (so either the current one, or a 32bit and a 64bit depending on the previous paragraph), bootstrap will extract the JSON spec of the target and change it to include `"panic-strategy": "abort"`. It will then build the standard library with this synthetic target, and bless the `mir-opt` suite with it.
As a result of these changes, blessing the `mir-opt` suite will actually bless it two or four times with different targets, ensuring all possible variants are actually blessed.
---
This PR is best reviewed commit-by-commit.
r? `@jyn514`
cc `@saethlin` `@oli-obk`
It might happen that a synthetic target name does not match one of the
hardcoded ones in std's build script, causing std to fail to build. This
commit changes the std build script avoid including the restricted-std
feature unconditionally when a synthetic target is being built.
Some targets are added to these hashmaps at runtime, and are not present
during dry runs. To avoid errors, this commit changes all the related
functions to always return empty strings/paths during dry runs.
This has real differences in the effective debuginfo: in particular, it omits the module-level information and breaks perf.
Allow passing `line-tables-only` directly in config.toml instead.
Rollup of 7 pull requests
Successful merges:
- #112475 (Fix issue for module name when surround the struct literal with parentheses)
- #112477 (Give more helpful progress messages in `Assemble`)
- #112484 (Fix ntdll linkage issues on Windows UWP platforms)
- #112492 (Migrate GUI colors test to original CSS color format)
- #112493 (iat selection: normalize self ty & completely erase bound vars)
- #112497 (abs_sub: fix typo 0[-:][+.]0)
- #112498 (Update links to Rust Reference in diagnostic)
r? `@ghost`
`@rustbot` modify labels: rollup
Give more helpful progress messages in `Assemble`
Before (download-rustc):
```
# no output
```
After (download-rustc):
```
Creating a sysroot for stage2 compiler (use `rustup toolchain link 'name' build/host/stage2`)
```
Before (compiling from source):
```
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
Assembling stage1 compiler
Build stage1 library artifacts (x86_64-unknown-linux-gnu -> i686-unknown-linux-gnu)
Building compiler artifacts (stage0:x86_64-unknown-linux-gnu -> stage1:i686-unknown-linux-gnu)
Assembling stage1 compiler (i686-unknown-linux-gnu)
```
After (compiling from source):
```
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
Creating a sysroot for stage1 compiler (use `rustup toolchain link 'name' build/host/stage1`)
Build stage1 library artifacts (x86_64-unknown-linux-gnu)
Building compiler artifacts (stage0:x86_64-unknown-linux-gnu -> stage1:i686-unknown-linux-gnu)
Creating a sysroot for stage1 compiler (i686-unknown-linux-gnu) (use `rustup toolchain link 'name' build/i686-unknown-linux-gnu/stage1`)
```
cc https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Unable.20to.20compile.20rustc.20MSVC, https://discord.com/channels/273534239310479360/957720175619215380/1116867245499498506
Don't compile rustc to self-test compiletest
This was changed from stage 0 to 1 in https://github.com/rust-lang/rust/pull/108905, but I'm not sure why. Change it to `top_stage` instead to allow people to choose the stage.
This should save quite a bit of time in the `mingw-check` builder, which explicitly runs `x test --stage 0 compiletest`.
Note that this also fixes a latent bug that depended on running `x build compiler` before `x doc compiler`, as well as a couple cleanups related to symlinks (which made the latent bug easier to find).
cc `@pietroalbini`
Make the build process more beginner friendly:
- Include information explaining that the stage2 toolchain should be
used (and not the stage1 toolchain) due to the `download-rustc`
setting.
- Display a message when the user runs `x setup tools` explaining that
they should use the stage2 toolchain.
bootstrap: Disallow `--exclude test::std`
Use the top-level Kind to determine whether Steps are excluded.
Previously, this would use the `Kind` passed to `--exclude` (and not do any filtering at all if no kind was passed).
That meant that `x test linkchecker --exclude std` would fail - you had to explicitly say `--exclude test::std`.
Change bootstrap to use the top-level Kind instead, which does the right thing automatically.
Note that this breaks things like `x test --exclude doc::std`, but I'm not sure why you'd ever want to do that.
There's a lot of churn here, but the 1-line change in the first commit is the actual behavior change, the rest is just cleanup.
Fixes https://github.com/rust-lang/rust/issues/103201. Note that this effectively reverts most of https://github.com/rust-lang/rust/pull/91965.
cc `@pietroalbini`
