Remove `ENABLE_DOWNLOAD_RUSTC` constant
`ENABLE_DOWNLOAD_RUSTC` was introduced as part of the MVP for `download-rustc` as a way not to rebuild artifacts that have already been downloaded. Unfortunately, it doesn't work very well:
- Steps are ignored by default, which makes it easy to leave out a step
that should be built. For example, the MVP forgot to enable any tests,
so it was only possible to *build* locally.
- It didn't work correctly even when it was enabled: calling
`builder.ensure()` would completely ignore the constant and rebuild the
step anyway. This has no obvious fix since `ensure()` has to return a
`Step::Output`.
Instead, this handles `download-rustc` in `impl Step for Rustc` and
`impl Step for Std`, which to my knowledge are the only build steps that
don't first go through `impl Step for Sysroot` (`Rustc` is used for
the `rustc-dev` component).
See https://github.com/rust-lang/rust/pull/79540#discussion_r563350075 and https://github.com/rust-lang/rust/issues/81930 for further context.
Here are some example runs with these changes and `download-rustc`
enabled:
```
$ x.py build src/tools/clippy
Building stage1 tool clippy-driver (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 1m 09s
Building stage1 tool cargo-clippy (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.11s
$ x.py test src/tools/clippy
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Building stage1 tool clippy-driver (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.09s
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.28s
Finished release [optimized] target(s) in 15.26s
Running build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-8b407b140e0aa91c
test result: ok. 592 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out
$ x.py build src/tools/rustdoc
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 41.28s
Build completed successfully in 0:00:41
$ x.py test src/test/rustdoc-ui
Building stage0 tool compiletest (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.12s
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.10s
test result: ok. 105 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.15s
$ x.py build compiler/rustc
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Build completed successfully in 0:00:00
```
Note a few things:
- Clippy depends on stage1 rustc-dev artifacts, but rustc didn't have to
be recompiled. Instead, the artifacts were copied automatically.
- All steps are always enabled. There is no danger of forgetting a step,
since only the entrypoints have to handle `download-rustc`.
- Building the compiler (`compiler/rustc`) automatically does no work.
Helps with https://github.com/rust-lang/rust/issues/81930.
r? `@Mark-Simulacrum`
config.toml parsing error improvements
Improve error messages for musl-libdir and wasi-root keys. Previously
the parser would panic with `unwrap()`. Now it prints
Target "wasm32-wasi" does not have a "wasi-root" key
(and similar for the `musl-libdir` field, which is used in target that
use musl)
Also update comments around wasi-root field to make it clear that the
field is only valid in wasm32-wasi target and needs to be moved to a
`[target.wasm32-wasi]` section to be valid.
Fixes#82317
---
r? `@Mark-Simulacrum`
Improve error messages for musl-libdir and wasi-root keys. Previously
the parser would panic with `unwrap()`. Now it prints
Target "wasm32-wasi" does not have a "wasi-root" key
(and similar for the `musl-libdir` field, which is used in target that
use musl)
Also update comments around wasi-root field to make it clear that the
field is only valid in wasm32-wasi target and needs to be moved to a
`[target.wasm32-wasi]` section to be valid.
Fixes#82317
Rename the `tidy` binary to `rust-tidy`
This avoids naming collisions, particularly on Windows where the
dynamic library variable is PATH and setting it causes the in-tree
`tidy` to take precedence over the HTML tidy used by compiletest.
This doesn't change the x.py interface in any way, it still accepts `x.py test tidy` and prints error messages about `tidy`. It only changes the name of the file on disk.
Fixes https://github.com/rust-lang/rust/issues/82501.
Propagate RUSTDOCFLAGS in the environment when documenting
Previously, RUSTDOCFLAGS would get overriden when bootstrap set
`RUSTDOCFLAGS` itself. Propagate the flag manually, using the same logic
as `RUSTFLAGS`.
Fixes https://github.com/rust-lang/rust/issues/75256.
For some targets, rustc uses a "CRT fallback", where it links CRT
object files it ships instead of letting the host compiler link
them.
On musl, rustc currently links crt1, crti and crtn (provided by
libc), but does not link crtbegin and crtend (provided by libgcc).
In particular, crtend is responsible for terminating the .eh_frame
section. Lack of terminator may result in segfaults during
unwinding, as reported in #47551 and encountered by the LLVM 12
update in #81451.
This patch links crtbegin and crtend for musl as well, following
the table at the top of crt_objects.rs.
On non-unix platforms, does not try to call `getrusage` (and does not attempt to
implement its own shim; that could be follow-on work, though its probably best
to not invest too much effort there, versus using separate dedicated tooling).
On unix platforms, calls libc::rusage and attempts to emit the subset of fields
that are supported on Linux and Mac OS X. Omits groups of related stats which
appear to be unsupported on the platform (due to them all remaining zero).
Adjusts output to compensate for Mac using bytes instead of kb (a well known
discrepancy on Mac OS X). However, so far I observe a lot of strange values
(orders of magnitude wrong) reported on Mac OS X in some cases, so I would not
trust this in that context currently.
This avoids naming collisions, particularly on Windows where the
dynamic library variable is PATH and setting it causes the in-tree
`tidy` to take precedence over the HTML tidy used by compiletest.
This was introduced as part of the MVP for `download-rustc`.
Unfortunately, it doesn't work very well:
- Steps are ignored by default, which makes it easy to leave out a step
that should be built. For example, the MVP forgot to enable any tests,
so it was *only* possible to build locally.
- It didn't work correctly even when it was enabled: calling
`builder.ensure()` would completely ignore the constant and rebuild the
step anyway. This has no obvious fix since `ensure()` has to return a
`Step::Output`.
Instead, this handles `download-rustc` in `impl Step for Rustc` and
`impl Step for Std`, which to my knowledge are the only build steps that
don't first go through `impl Step for Sysroot` (`Rustc` is used for
the `rustc-dev` component).
See https://github.com/rust-lang/rust/pull/79540#discussion_r563350075
and https://github.com/rust-lang/rust/issues/81930 for further context.
Here are some example runs with these changes and `download-rustc`
enabled:
```
$ x.py build src/tools/clippy
Building stage1 tool clippy-driver (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 1m 09s
Building stage1 tool cargo-clippy (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.11s
$ x.py test src/tools/clippy
Updating only changed submodules
Submodules updated in 0.01 seconds
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Building stage1 tool clippy-driver (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.09s
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.28s
Finished release [optimized] target(s) in 15.26s
Running build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-8b407b140e0aa91c
test result: ok. 592 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out
$ x.py build src/tools/rustdoc
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 41.28s
Build completed successfully in 0:00:41
$ x.py test src/test/rustdoc-ui
Building stage0 tool compiletest (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.12s
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.10s
test result: ok. 105 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.15s
$ x.py build compiler/rustc
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Build completed successfully in 0:00:00
```
Note a few things:
- Clippy depends on stage1 rustc-dev artifacts, but rustc didn't have to
be recompiled. Instead, the artifacts were copied automatically.
- All steps are always enabled. There is no danger of forgetting a step,
since only the entrypoints have to handle `download-rustc`.
- Building the compiler (`compiler/rustc`) automatically does no work.
Update the bootstrap compiler
This updates the bootstrap compiler, notably leaving out a change to enable semicolon in macro expressions lint, because stdarch still depends on the old behavior.
Rustdoc gui tests
This is a reopening of #70533.
For this first version, there will be no screenshot comparison. Also, a big change compared to the previous version: the tests are now hosted in the rust repository directly. Since there is no image, it's pretty lightweight to say the least.
So now, only remains the nodejs script to run the tests and the tests themselves. Just one thing is missing: where should I put the documentation for these tests? I'm not sure where would be the best place for that. The doc will contain important information like the documentation of the framework used and how to install it (`npm install browser-ui-test`, but still needs to be put somewhere so no one is lost).
We'd also need to install the package when running the CI too. For now, it runs as long as we have nodejs installed, but I think we don't it to run in all nodejs targets?
cc `@jyn514`
r? `@Mark-Simulacrum`
Previously, RUSTDOCFLAGS would get overriden when bootstrap set
`RUSTDOCFLAGS` itself. Propagate the flag manually, using the same logic
as `RUSTFLAGS`.
This also extracts the logic into a helper function to make sure it's
the same.
Do not delete bootstrap.exe on Windows during clean
Windows does not allow deleting currently running executables.
This an addition to ```@jyn514's``` change in https://github.com/rust-lang/rust/pull/80574.
parallelize x.py test tidy
Running tidy on individual commits when rewriting git history was somewhat of an annoyance, so I have parallelized it a bit.
running `time ./x.py test tidy` with warm IO caches:
old:
```
real 0m11.123s
user 0m14.495s
sys 0m5.227s
```
new:
```
real 0m1.834s
user 0m13.545s
sys 0m3.094s
```
There's further room for improvement (<0.9s should be feasible) but that would require bigger changes.
std_detect is still using this and as it's in a submodule updating it will be a
pain. We can catch this either after a stdarch submodule bump or just on the
next cycle.
Make sure pdbs are copied along with exe and dlls when bootstrapping
This makes it easier to find the pdbs when wanting to debug the compiler on Windows.
make `suggest_setup` help messages better
When I first built the compiler and didn't create a `config.toml.example`, the following was emitted:
```
help: consider running `x.py setup` or copying `config.toml.example`
```
I ran `x.py setup` but got an error so in this PR I made the help messages a little clearer.
avoid full-slicing slices
If we already have a slice, there is no need to get another full-range slice from that, just use the original.
clippy::redundant_slicing
Remove unnecessary `Option` in `default_doc`
Previously, there were two different ways to encode the same info: `None` or
`Some(&[])`. Now there is only one way, `&[]`.
dist: include src/build_helper as part of the crate graph for rustc-dev
The build_helper dependency is used to build the compiler/rustc_llvm build script.
Since it was missing, it wasn't possible to really use rustc-dev to
build, see for instance: https://github.com/rust-analyzer/rust-analyzer/issues/7589.
bootstrap: Locate llvm-dwp based on llvm-config bindir
Fixes#81949.
Tested by successfully building 1.50.0 pre-release, which is where I originally hit the issue (https://internals.rust-lang.org/t/rust-1-50-0-pre-release-testing/14012/4?u=dtolnay). Tested both with and without prebuilt LLVM. The check for dry_run is necessary in the non-prebuilt case because the llvm-config built by bootstrap won't exist yet.
