Rename the `tidy` binary to `rust-tidy`
This avoids naming collisions, particularly on Windows where the
dynamic library variable is PATH and setting it causes the in-tree
`tidy` to take precedence over the HTML tidy used by compiletest.
This doesn't change the x.py interface in any way, it still accepts `x.py test tidy` and prints error messages about `tidy`. It only changes the name of the file on disk.
Fixes https://github.com/rust-lang/rust/issues/82501.
Propagate RUSTDOCFLAGS in the environment when documenting
Previously, RUSTDOCFLAGS would get overriden when bootstrap set
`RUSTDOCFLAGS` itself. Propagate the flag manually, using the same logic
as `RUSTFLAGS`.
Fixes https://github.com/rust-lang/rust/issues/75256.
For some targets, rustc uses a "CRT fallback", where it links CRT
object files it ships instead of letting the host compiler link
them.
On musl, rustc currently links crt1, crti and crtn (provided by
libc), but does not link crtbegin and crtend (provided by libgcc).
In particular, crtend is responsible for terminating the .eh_frame
section. Lack of terminator may result in segfaults during
unwinding, as reported in #47551 and encountered by the LLVM 12
update in #81451.
This patch links crtbegin and crtend for musl as well, following
the table at the top of crt_objects.rs.
On non-unix platforms, does not try to call `getrusage` (and does not attempt to
implement its own shim; that could be follow-on work, though its probably best
to not invest too much effort there, versus using separate dedicated tooling).
On unix platforms, calls libc::rusage and attempts to emit the subset of fields
that are supported on Linux and Mac OS X. Omits groups of related stats which
appear to be unsupported on the platform (due to them all remaining zero).
Adjusts output to compensate for Mac using bytes instead of kb (a well known
discrepancy on Mac OS X). However, so far I observe a lot of strange values
(orders of magnitude wrong) reported on Mac OS X in some cases, so I would not
trust this in that context currently.
This avoids naming collisions, particularly on Windows where the
dynamic library variable is PATH and setting it causes the in-tree
`tidy` to take precedence over the HTML tidy used by compiletest.
Update the bootstrap compiler
This updates the bootstrap compiler, notably leaving out a change to enable semicolon in macro expressions lint, because stdarch still depends on the old behavior.
Rustdoc gui tests
This is a reopening of #70533.
For this first version, there will be no screenshot comparison. Also, a big change compared to the previous version: the tests are now hosted in the rust repository directly. Since there is no image, it's pretty lightweight to say the least.
So now, only remains the nodejs script to run the tests and the tests themselves. Just one thing is missing: where should I put the documentation for these tests? I'm not sure where would be the best place for that. The doc will contain important information like the documentation of the framework used and how to install it (`npm install browser-ui-test`, but still needs to be put somewhere so no one is lost).
We'd also need to install the package when running the CI too. For now, it runs as long as we have nodejs installed, but I think we don't it to run in all nodejs targets?
cc `@jyn514`
r? `@Mark-Simulacrum`
Previously, RUSTDOCFLAGS would get overriden when bootstrap set
`RUSTDOCFLAGS` itself. Propagate the flag manually, using the same logic
as `RUSTFLAGS`.
This also extracts the logic into a helper function to make sure it's
the same.
Do not delete bootstrap.exe on Windows during clean
Windows does not allow deleting currently running executables.
This an addition to ```@jyn514's``` change in https://github.com/rust-lang/rust/pull/80574.
parallelize x.py test tidy
Running tidy on individual commits when rewriting git history was somewhat of an annoyance, so I have parallelized it a bit.
running `time ./x.py test tidy` with warm IO caches:
old:
```
real 0m11.123s
user 0m14.495s
sys 0m5.227s
```
new:
```
real 0m1.834s
user 0m13.545s
sys 0m3.094s
```
There's further room for improvement (<0.9s should be feasible) but that would require bigger changes.
std_detect is still using this and as it's in a submodule updating it will be a
pain. We can catch this either after a stdarch submodule bump or just on the
next cycle.
Make sure pdbs are copied along with exe and dlls when bootstrapping
This makes it easier to find the pdbs when wanting to debug the compiler on Windows.
make `suggest_setup` help messages better
When I first built the compiler and didn't create a `config.toml.example`, the following was emitted:
```
help: consider running `x.py setup` or copying `config.toml.example`
```
I ran `x.py setup` but got an error so in this PR I made the help messages a little clearer.
avoid full-slicing slices
If we already have a slice, there is no need to get another full-range slice from that, just use the original.
clippy::redundant_slicing
Remove unnecessary `Option` in `default_doc`
Previously, there were two different ways to encode the same info: `None` or
`Some(&[])`. Now there is only one way, `&[]`.
dist: include src/build_helper as part of the crate graph for rustc-dev
The build_helper dependency is used to build the compiler/rustc_llvm build script.
Since it was missing, it wasn't possible to really use rustc-dev to
build, see for instance: https://github.com/rust-analyzer/rust-analyzer/issues/7589.
bootstrap: Locate llvm-dwp based on llvm-config bindir
Fixes#81949.
Tested by successfully building 1.50.0 pre-release, which is where I originally hit the issue (https://internals.rust-lang.org/t/rust-1-50-0-pre-release-testing/14012/4?u=dtolnay). Tested both with and without prebuilt LLVM. The check for dry_run is necessary in the non-prebuilt case because the llvm-config built by bootstrap won't exist yet.
Use format string in bootstrap panic instead of a string directly
This fixes the following warning when compiling with nightly:
```
warning: panic message is not a string literal
--> src/bootstrap/builder.rs:1515:24
|
1515 | panic!(out);
| ^^^
|
= note: `#[warn(non_fmt_panic)]` on by default
= note: this is no longer accepted in Rust 2021
help: add a "{}" format string to Display the message
|
1515 | panic!("{}", out);
| ^^^^^
help: or use std::panic::panic_any instead
|
1515 | std::panic::panic_any(out);
| ^^^^^^^^^^^^^^^^^^^^^^
```
Found while working on https://github.com/rust-lang/rust/pull/79540. cc https://github.com/rust-lang/rust/pull/81645, which landed in 1.51.
Allow building rustdoc without first building rustc (MVP)
## Motivation
The compile times for rustc are extremely long and a major issue for
recruiting new contributors to rustdoc. People interested in joining
often give up after running into issues with submodules or python
versions. stage1 rustdoc fundamentally doesn't care about bootstrapping
or stages, it just needs `rustc_private` available.
## Summary of Changes
- Add an opt-in `[rust] download_rustc` option
- Determine the version of the compiler to download using `log --author=bors`
- Do no work for any component other than `Rustdoc` for any stage. Instead, copy the CI artifacts from the downloaded sysroot stage0/ to stage0-sysroot/ or stage1/ in `Sysroot`. This is done with an `ENABLE_DOWNLOAD_STAGE1` constant which is off by default.
- Don't download different versions of rustfmt or cargo - those should still use the beta version (rustfmt especially).
The vast majority of work is done in bootstrap.py, which downloads the artifacts and extracts them to stage0/ in place of the beta compiler. Rustbuild just takes care of copying the artifacts to stage1 if necessary.
## Future work
- I turned off verification for the commit tarballs because the .sha256 URLs gave a 404. This seems not ideal, it would be nice to start signing them.
- This will break if you rebase an old enough branch (I think commits are kept at most 160 days?). This doesn't need to be supported, but it would be nice to give a reasonable error. https://github.com/rust-lang/rust/pull/79540#issuecomment-751481291
- Right now, every time you rebase, stage0 tools (bootstrap, tidy, ...) will have to be recompiled. Additionally running `x.py setup tools` will compile rustbuild twice. Instead, this should download a separate beta compiler for stage0 and only use CI artifacts for stage1 onward. https://github.com/rust-lang/rust/pull/79540#issuecomment-757047321
- Add `x.py setup tools` to enable this conveniently (it doesn't make sense to use this for compiler developers). cb5d8c8522
- Compile a new version of tracing so that rustdoc still gets debug logging (since CI artifacts always disable `debug` and `trace` logging). https://github.com/rust-lang/rust/pull/79540#issuecomment-742756411, 6a5d512420
- Right now only rustdoc is ever rebuilt. This is not ideal and should probably at least compile compiler tools (rustfmt, clippy, miri). https://github.com/rust-lang/rust/pull/79540#issuecomment-775634693
- Using `git log --author=bors` sometimes breaks. This should use `git merge-base` instead. https://github.com/rust-lang/rust/pull/79540#discussion_r572572280
- It would be nice to support cross-compiling the standard library. Right now this gives an assertion failure I think.
Some of this work has already been done in (the history for) 673476c785.
- Use the same compiler for stage0 and stage1. This should be fixed at
some point (so bootstrap isn't constantly rebuilt).
- Make sure `x.py build` and `x.py check` work.
- Use `git merge-base` to determine the most recent commit to download.
- Copy stage0 to the various sysroots in `Sysroot`, and delegate to
Sysroot in Assemble. Leave all other code unchanged.
- Rename date -> key
This can also be a commit hash, so 'date' is no longer a good name.
- Add the commented-out option to config.toml.example
- Disable all steps by default when `download-rustc` is enabled
Most steps don't make sense when downloading a compiler, because they'll
be pre-built in the sysroot. Only enable the ones that might be useful,
in particular Rustdoc and all `check` steps.
At some point, this should probably enable other tools, but rustdoc is
enough to test out `download-rustc`.
- Don't print 'Skipping' twice in a row
Bootstrap forcibly enables a dry run if it isn't already set, so
previously it would print the message twice:
```
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
```
Now it correctly only prints once.
## Future work
- Add FIXME about supporting beta commits
- Debug logging will never work. This should be fixed.
This fixes the following warning when compiling with nightly:
```
warning: panic message is not a string literal
--> src/bootstrap/builder.rs:1515:24
|
1515 | panic!(out);
| ^^^
|
= note: `#[warn(non_fmt_panic)]` on by default
= note: this is no longer accepted in Rust 2021
help: add a "{}" format string to Display the message
|
1515 | panic!("{}", out);
| ^^^^^
help: or use std::panic::panic_any instead
|
1515 | std::panic::panic_any(out);
| ^^^^^^^^^^^^^^^^^^^^^^
```
