Fix running rustdoc-js test suite individually
Without `Compiletest.path` set running `x.py test src/test/rustdoc-js` would run the `rustdoc-js` test suite with everything filtered out.
As this was the only place setting `Compiletest.path` to `None` this removes the `Option` wrapper as well.
Rather than a boolean `--enable-cflags` etc., these options should
reflect that they are for LLVM, and that they need a value. You would
now use `./configure --llvm-cflags="..."`.
Without `Compiletest.path` set running `x.py test src/test/rustdoc-js` would run the `rustdoc-js` test suite with everything filtered out.
As this was the only place setting `Compiletest.path` to `None` this removes the `Option` wrapper as well.
Update books.
This required some changes in how the books are tested due to some changes in rust-lang/book. It uses new syntax that is not compatible with bare `rustdoc --test`. This changes it so that it uses rustbook to run the tests, which is essentially the same as `mdbook test`.
## reference
7 commits in 11e893fc1357bc688418ddf1087c2b7aa25d154d..64239df6d173562b9deb4f012e4c3e6e960c4754
2020-01-18 21:24:08 +0100 to 2020-02-10 19:05:13 +0100
- Update for nested receivers. (rust-lang-nursery/reference#724)
- clarify note re. leading `::` in 2018 (rust-lang-nursery/reference#752)
- Update macro-ambiguity.md (rust-lang-nursery/reference#754)
- typo fix: add missing `by` (rust-lang-nursery/reference#753)
- fix `TypeParamBounds` link on trait objects (rust-lang-nursery/reference#749)
- reorganize docs on references (rust-lang-nursery/reference#745)
- add MacroRepOp usage for ? (rust-lang-nursery/reference#744)
## book
49 commits in 87dd6843678575f8dda962f239d14ef4be14b352..6fb3705e5230311b096d47f7e2c91f9ce24393d0
2020-01-20 15:20:40 -0500 to 2020-02-12 13:48:57 -0500
- Fix nomicon links. (rust-lang/book#2253)
- Update to Rust 1.41.0 (rust-lang/book#2244)
- Listing 19-6: use ptr.add instead of ptr.offset (rust-lang/book#2201)
- Remove unneeded mutable reference
- Clarify deref coercion explanation
- Fix typo in link to 1.30 book
- Acknowledge Murphy's Law
- Clarify that buffer overread is UB in C
- Change from "must" to "idiomatic" about comments
- Fancy quotes
- Make HashMap types match previous example; add fwd ref to ch 13
- Tweak wording to array clarification
- Merge remote-tracking branch 'origin/pr/2236'
- Update all our crates (rust-lang/book#2235)
- Reword git caveat
- Merge remote-tracking branch 'origin/pr/2234'
- Merge remote-tracking branch 'origin/pr/2230'
- println! is a macro (rust-lang/book#2224)
- Update a translated version link (rust-lang/book#2221)
- move `Macro invocation` from section on tuple to section on mac… (rust-lang/book#2206)
- Do not limit `Self` usage in trait implementation (rust-lang/book#2197)
- Merge remote-tracking branch 'origin/pr/2191'
- Fix wrapping
- Merge remote-tracking branch 'origin/pr/2187'
- Updated appendix 07 to reflect deprecation of rustup install (rust-lang/book#2181)
- Make links to the Nomicon consistent
- Merge remote-tracking branch 'origin/pr/2180'
- Merge remote-tracking branch 'origin/pr/2175'
- Merge remote-tracking branch 'origin/pr/2171'
- Merge remote-tracking branch 'origin/pr/2170'
- Clarify and make consistent the explanation of unions
- Merge remote-tracking branch 'origin/pr/2166'
- Handle dev or test in the Finished output line
- Link to macros by example rather than macros (rust-lang/book#2164)
- Merge remote-tracking branch 'origin/pr/2147'
- Fix parens (rust-lang/book#2132)
- Clarify type inference with closures requires calling the closures
- Update link to French translation (rust-lang/book#2119)
- Merge remote-tracking branch 'origin/pr/2108'
- Add an explicit cross reference to data type
- Merge remote-tracking branch 'origin/pr/2105'
- ch15-02-deref: Improve explanation on immut-to-mut (rust-lang/book#2030)
- Remove unnecessary quotes
- Make markdown link identifier match
- Remove extra newline
- Merge remote-tracking branch 'origin/pr/2004'
- Extract code and output; script formatting and updating them (rust-lang/book#2231)
- Switch "Finally" to "Next" to reflect new chapters having been… (rust-lang/book#2098)
- ch19-06 added curly braces to macro output (rust-lang/book#2050)
## rust-by-example
2 commits in 1c2bd024d13f8011307e13386cf1fea2180352b5..32facd5522ddbbf37baf01e4e4b6562bc55c071a
2020-01-20 12:18:36 -0300 to 2020-02-11 09:25:06 -0300
- Add missing `dyn` in code sample (rust-lang/rust-by-example#1306)
- Improve grammar in a few sections (rust-lang/rust-by-example#1305)
## edition-guide
1 commits in 1a2390247ad6d08160e0dd74f40a01a9578659c2..37f9e6848411188a1062ead1bd8ebe4b8aa16899
2019-12-29 10:40:55 -0800 to 2020-02-10 14:36:14 +0100
- Fixed typo (rust-lang-nursery/edition-guide#196)
## embedded-book
4 commits in 4d78994915af1bde9a95c04a8c27d8dca066232a..b2e1092bf67bd4d7686c4553f186edbb7f5f92db
2020-01-14 08:25:25 +0000 to 2020-01-30 08:45:46 +0000
- Make typestate initialization notes correct (rust-embedded/book#224)
- Mention discovery book more prominently (rust-embedded/book#219)
- Replace nursery links with rust-lang links (rust-embedded/book#222)
- Add a Glossary appendix page (rust-embedded/book#223)
bootstrap: Configure cmake when building sanitizer runtimes
Configure cmake before building sanitizer runtimes in similar way it is already
configured elsewhere, to ensure that they are built with expected compiler
flags.
Previously this step has been intentionally omitted since sanitizer runtimes
are built as universal binaries on Darwin targets, which in turn are
unsupported by sccache which is also configured there. To avoid the issue
everything but the compiler launcher is configured.
Helps with #68863.
rustbuild: include channel in sanitizers installed name
Allows parallel install of different rust channels.
I'm not sure if the channel is the right thing to use there, but currently both beta and nightly try to install e.g. `/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_rt.asan.a` when before (and in current stable) it used to be `/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_asan-45a4390180e83d28.rlib` which contained a hash, making it unique.
With this patch, `/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.asan.a` gets installed
Python script PEP8 style guide space formatting and minor Python source cleanup
This PR includes the following changes in the Python sources based on a flake8 3.7.9 (mccabe: 0.6.1, pycodestyle: 2.5.0, pyflakes: 2.1.1) CPython 3.7.6 on Darwin lint:
- PEP8 style guide spacing updates *without* line length changes
- removal of unused local variable assignments in context managers and exception handling
- removal of unused Python import statements
- removal of unnecessary semicolons
Enable Control Flow Guard in rustbuild
Now that Rust supports Control Flow Guard (#68180), add a config.toml option to build the standard library with CFG enabled.
r? @nagisa
Remove unused feature gates
I think many of the remaining unstable things can be easily be replaced with stable things. I have kept the `#![feature(nll)]` even though it is only necessary in `libstd`, to make regressions of it harder.
Add an option to use LLD to link the compiler on Windows platforms
Based on https://github.com/rust-lang/rust/pull/68609.
Using LLD is good way to improve compile times on Windows since `link.exe` is quite slow. The time for `x.py build --stage 1 src/libtest` goes from 0:12:00 to 0:08:29. Compile time for `rustc_driver` goes from 226.34s to 18.5s. `rustc_macros` goes from 28.69s to 7.7s. The size of `rustc_driver` is also reduced from 83.3 MB to 78.7 MB.
r? @Mark-Simulacrum
Step stage0 to bootstrap from 1.42
This also includes a commit which fixes the rustfmt downloading logic to redownload when the rustfmt channel changes, and bumps rustfmt to a more recent version.
Clear out std, not std tools
This was a typo that slipped in, and meant that we were still not properly
clearing out std.
This is basically #67760 but actually correct...
Enable ASan on Fuchsia
This change adds the x86_64-fuchsia and aarch64-fuchsia LLVM targets to
those allowed to invoke -Zsanitizer. Currently, the only overlap between
compiler_rt sanitizers supported by both rustc and Fuchsia is ASan.
compiletest: Simplify multi-debugger support
Previous implementation used a single mode type to store various pieces
of otherwise loosely related information:
* Whether debuginfo mode is in use or not.
* Which debuggers should run in general.
* Which debuggers are enabled for particular test case.
The new implementation introduces a separation between those aspects.
There is a single debuginfo mode parametrized by a debugger type.
The debugger detection is performed first and a separate configuration
is created for each detected debugger. The test cases are gathered
independently for each debugger which makes it trivial to implement
support for `ignore` / `only` conditions.
Functional changes:
* A single `debuginfo` entry point (rather than `debuginfo-cdb`, `debuginfo-gdb+lldb`, etc.).
* Debugger name is included in the test name.
* Test outputs are placed in per-debugger directory.
* Fixed spurious hash mismatch. Previously, the config mode would change
from `DebugInfoGdbLldb` (when collecting tests) to `DebugInfoGdb` or
`DebugInfoLldb` (when running them) which would affect hash computation.
* PYTHONPATH is additionally included in gdb hash.
* lldb-python and lldb-python-dir are additionally included in lldb hash.
This change adds the x86_64-fuchsia and aarch64-fuchsia LLVM targets to
those allowed to invoke -Zsanitizer. Currently, the only overlap between
compiler_rt sanitizers supported by both rustc and Fuchsia is ASan.
Clarify where the clippy used in `./x.py clippy` is coming from.
It uses whatever clippy binary was installed via rustup, cargo-install
or otherwise and does NOT use the binary generated by `./x.py build src/tools/clippy`.
Previous implementation used a single mode type to store various pieces
of otherwise loosely related information:
* Whether debuginfo mode is in use or not.
* Which debuggers should run in general.
* Which debuggers are enabled for particular test case.
The new implementation introduces a separation between those aspects.
There is a single debuginfo mode parametrized by a debugger type.
The debugger detection is performed first and a separate configuration
is created for each detected debugger. The test cases are gathered
independently for each debugger which makes it trivial to implement
support for `ignore` / `only` conditions.
Functional changes:
* A single `debuginfo` entry point (rather than `debuginfo-cdb`, `debuginfo-gdb+lldb`, etc.).
* Debugger name is included in the test name.
* Test outputs are placed in per-debugger directory.
* Fixed spurious hash mismatch. Previously, the config mode would change
from `DebugInfoGdbLldb` (when collecting tests) to `DebugInfoGdb` or
`DebugInfoLldb` (when running them) which would affect hash computation.
* PYTHONPATH is additionally included in gdb hash.
* lldb-python and lldb-python-dir are additionally included in lldb hash.
Better support for cross compilation on Windows.
I have been investigating enabling panic=unwind for aarch64-pc-windows-msvc (see #65313) and building rustc and cargo hosted on aarch64-pc-windows-msvc.
Without the libpath changes we were trying to link a mix of amd64 and arm64 binaries.
Without the cmake system name change, the llvm build was trying to run an arm64 build tool on the x86_64 build machine.
That said, I haven't tested all different combinations here and am very open to resolving this a different way.
Add `llvm-skip-rebuild` flag to `x.py`
This PR follows on from #67437 to complete the feature request from #65612.
Specifically it adds a new command-line flag, `--llvm-skip-rebuild`, which overrides both any value set in `config.toml` and the default value (`false`).
I'm not 100% confident that I've implemented the override in the "best" way, but I've checked it locally and it seems to work at least.
This option isn't currently mentioned in the Guide to Rustc Development. I'd be happy to write something on it if folk think that's worthwhile.
build-std compatible sanitizer support
### Motivation
When using `-Z sanitizer=*` feature it is essential that both user code and
standard library is instrumented. Otherwise the utility of sanitizer will be
limited, or its use will be impractical like in the case of memory sanitizer.
The recently introduced cargo feature build-std makes it possible to rebuild
standard library with arbitrary rustc flags. Unfortunately, those changes alone
do not make it easy to rebuild standard library with sanitizers, since runtimes
are dependencies of std that have to be build in specific environment,
generally not available outside rustbuild process. Additionally rebuilding them
requires presence of llvm-config and compiler-rt sources.
The goal of changes proposed here is to make it possible to avoid rebuilding
sanitizer runtimes when rebuilding the std, thus making it possible to
instrument standard library for use with sanitizer with simple, although
verbose command:
```
env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu
```
### Implementation
* Sanitizer runtimes are no long packed into crates. Instead, libraries build
from compiler-rt are used as is, after renaming them into `librusc_rt.*`.
* rustc obtains runtimes from target libdir for default sysroot, so that
they are not required in custom build sysroots created with build-std.
* The runtimes are only linked-in into executables to address issue #64629.
(in previous design it was hard to avoid linking runtimes into static
libraries produced by rustc as demonstrated by sanitizer-staticlib-link
test, which still passes despite changes made in #64780).
cc @kennytm, @japaric, @firstyear, @choller
