bootstrap: Configure cmake when building sanitizer runtimes
Configure cmake before building sanitizer runtimes in similar way it is already
configured elsewhere, to ensure that they are built with expected compiler
flags.
Previously this step has been intentionally omitted since sanitizer runtimes
are built as universal binaries on Darwin targets, which in turn are
unsupported by sccache which is also configured there. To avoid the issue
everything but the compiler launcher is configured.
Helps with #68863.
rustbuild: include channel in sanitizers installed name
Allows parallel install of different rust channels.
I'm not sure if the channel is the right thing to use there, but currently both beta and nightly try to install e.g. `/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_rt.asan.a` when before (and in current stable) it used to be `/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_asan-45a4390180e83d28.rlib` which contained a hash, making it unique.
With this patch, `/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.asan.a` gets installed
Python script PEP8 style guide space formatting and minor Python source cleanup
This PR includes the following changes in the Python sources based on a flake8 3.7.9 (mccabe: 0.6.1, pycodestyle: 2.5.0, pyflakes: 2.1.1) CPython 3.7.6 on Darwin lint:
- PEP8 style guide spacing updates *without* line length changes
- removal of unused local variable assignments in context managers and exception handling
- removal of unused Python import statements
- removal of unnecessary semicolons
Enable Control Flow Guard in rustbuild
Now that Rust supports Control Flow Guard (#68180), add a config.toml option to build the standard library with CFG enabled.
r? @nagisa
Remove unused feature gates
I think many of the remaining unstable things can be easily be replaced with stable things. I have kept the `#![feature(nll)]` even though it is only necessary in `libstd`, to make regressions of it harder.
Add an option to use LLD to link the compiler on Windows platforms
Based on https://github.com/rust-lang/rust/pull/68609.
Using LLD is good way to improve compile times on Windows since `link.exe` is quite slow. The time for `x.py build --stage 1 src/libtest` goes from 0:12:00 to 0:08:29. Compile time for `rustc_driver` goes from 226.34s to 18.5s. `rustc_macros` goes from 28.69s to 7.7s. The size of `rustc_driver` is also reduced from 83.3 MB to 78.7 MB.
r? @Mark-Simulacrum
Step stage0 to bootstrap from 1.42
This also includes a commit which fixes the rustfmt downloading logic to redownload when the rustfmt channel changes, and bumps rustfmt to a more recent version.
Clear out std, not std tools
This was a typo that slipped in, and meant that we were still not properly
clearing out std.
This is basically #67760 but actually correct...
Enable ASan on Fuchsia
This change adds the x86_64-fuchsia and aarch64-fuchsia LLVM targets to
those allowed to invoke -Zsanitizer. Currently, the only overlap between
compiler_rt sanitizers supported by both rustc and Fuchsia is ASan.
compiletest: Simplify multi-debugger support
Previous implementation used a single mode type to store various pieces
of otherwise loosely related information:
* Whether debuginfo mode is in use or not.
* Which debuggers should run in general.
* Which debuggers are enabled for particular test case.
The new implementation introduces a separation between those aspects.
There is a single debuginfo mode parametrized by a debugger type.
The debugger detection is performed first and a separate configuration
is created for each detected debugger. The test cases are gathered
independently for each debugger which makes it trivial to implement
support for `ignore` / `only` conditions.
Functional changes:
* A single `debuginfo` entry point (rather than `debuginfo-cdb`, `debuginfo-gdb+lldb`, etc.).
* Debugger name is included in the test name.
* Test outputs are placed in per-debugger directory.
* Fixed spurious hash mismatch. Previously, the config mode would change
from `DebugInfoGdbLldb` (when collecting tests) to `DebugInfoGdb` or
`DebugInfoLldb` (when running them) which would affect hash computation.
* PYTHONPATH is additionally included in gdb hash.
* lldb-python and lldb-python-dir are additionally included in lldb hash.
This change adds the x86_64-fuchsia and aarch64-fuchsia LLVM targets to
those allowed to invoke -Zsanitizer. Currently, the only overlap between
compiler_rt sanitizers supported by both rustc and Fuchsia is ASan.
Clarify where the clippy used in `./x.py clippy` is coming from.
It uses whatever clippy binary was installed via rustup, cargo-install
or otherwise and does NOT use the binary generated by `./x.py build src/tools/clippy`.
Previous implementation used a single mode type to store various pieces
of otherwise loosely related information:
* Whether debuginfo mode is in use or not.
* Which debuggers should run in general.
* Which debuggers are enabled for particular test case.
The new implementation introduces a separation between those aspects.
There is a single debuginfo mode parametrized by a debugger type.
The debugger detection is performed first and a separate configuration
is created for each detected debugger. The test cases are gathered
independently for each debugger which makes it trivial to implement
support for `ignore` / `only` conditions.
Functional changes:
* A single `debuginfo` entry point (rather than `debuginfo-cdb`, `debuginfo-gdb+lldb`, etc.).
* Debugger name is included in the test name.
* Test outputs are placed in per-debugger directory.
* Fixed spurious hash mismatch. Previously, the config mode would change
from `DebugInfoGdbLldb` (when collecting tests) to `DebugInfoGdb` or
`DebugInfoLldb` (when running them) which would affect hash computation.
* PYTHONPATH is additionally included in gdb hash.
* lldb-python and lldb-python-dir are additionally included in lldb hash.
Better support for cross compilation on Windows.
I have been investigating enabling panic=unwind for aarch64-pc-windows-msvc (see #65313) and building rustc and cargo hosted on aarch64-pc-windows-msvc.
Without the libpath changes we were trying to link a mix of amd64 and arm64 binaries.
Without the cmake system name change, the llvm build was trying to run an arm64 build tool on the x86_64 build machine.
That said, I haven't tested all different combinations here and am very open to resolving this a different way.
Add `llvm-skip-rebuild` flag to `x.py`
This PR follows on from #67437 to complete the feature request from #65612.
Specifically it adds a new command-line flag, `--llvm-skip-rebuild`, which overrides both any value set in `config.toml` and the default value (`false`).
I'm not 100% confident that I've implemented the override in the "best" way, but I've checked it locally and it seems to work at least.
This option isn't currently mentioned in the Guide to Rustc Development. I'd be happy to write something on it if folk think that's worthwhile.
build-std compatible sanitizer support
### Motivation
When using `-Z sanitizer=*` feature it is essential that both user code and
standard library is instrumented. Otherwise the utility of sanitizer will be
limited, or its use will be impractical like in the case of memory sanitizer.
The recently introduced cargo feature build-std makes it possible to rebuild
standard library with arbitrary rustc flags. Unfortunately, those changes alone
do not make it easy to rebuild standard library with sanitizers, since runtimes
are dependencies of std that have to be build in specific environment,
generally not available outside rustbuild process. Additionally rebuilding them
requires presence of llvm-config and compiler-rt sources.
The goal of changes proposed here is to make it possible to avoid rebuilding
sanitizer runtimes when rebuilding the std, thus making it possible to
instrument standard library for use with sanitizer with simple, although
verbose command:
```
env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu
```
### Implementation
* Sanitizer runtimes are no long packed into crates. Instead, libraries build
from compiler-rt are used as is, after renaming them into `librusc_rt.*`.
* rustc obtains runtimes from target libdir for default sysroot, so that
they are not required in custom build sysroots created with build-std.
* The runtimes are only linked-in into executables to address issue #64629.
(in previous design it was hard to avoid linking runtimes into static
libraries produced by rustc as demonstrated by sanitizer-staticlib-link
test, which still passes despite changes made in #64780).
cc @kennytm, @japaric, @firstyear, @choller
remove explicit strip-hidden pass from compiler doc generation
`strip-hidden` is now implied by `--document-private-items` with #67875, so there's no need to specify it anymore.
