Add aarch64-unknown-linux-musl host builds
This adds aarch64-unknown-linux-musl to the hosts list and adds the build to the dist-arm-linux builder as `@Mark-Simulacrum` suggested to me in Zulip. `@jyn514` requested to be mentioned 😄
I had to update the config for crosstool-ng as it had a prompt about the glibc version.
I ran `src/ci/docker/run.sh dist-arm-linux` to test it.
```
Build completed successfully in 1:31:50
Compile requests 8180
Compile requests executed 8135
Cache hits 287
Cache misses 7848
Cache timeouts 0
Cache read errors 0
Forced recaches 0
Cache write errors 0
Compilation failures 0
Cache errors 0
Non-cacheable compilations 0
Non-cacheable calls 36
Non-compilation calls 9
Unsupported compiler calls 0
Average cache write 0.000 s
Average cache read miss 6.389 s
Average cache read hit 0.000 s
Cache location Local disk: "/sccache"
Cache size 202 MiB
Max cache size 10 GiB
== clock drift check ==
local time: Sun Sep 6 19:30:17 UTC 2020
network time: Sun, 06 Sep 2020 19:30:17 GMT
== end clock drift check ==
```
Only errors were in miri due to struct fields being private (already been reported [here](https://github.com/rust-lang/rust/issues/76337))
Edit: Maybe it is helpful if I add that it is a working compiler
```sh
/rust-nightly-aarch64-unknown-linux-musl # ash install.sh
install: creating uninstall script at /usr/local/lib/rustlib/uninstall.sh
install: installing component 'rustc'
install: installing component 'cargo'
install: installing component 'rls-preview'
install: installing component 'rust-analyzer-preview'
install: installing component 'clippy-preview'
install: installing component 'rustfmt-preview'
install: installing component 'llvm-tools-preview'
install: installing component 'rust-analysis-aarch64-unknown-linux-musl'
install: installing component 'rust-std-aarch64-unknown-linux-musl'
install: WARNING: failed to run ldconfig. this may happen when not installing as root. run with --verbose to see the error
Rust is ready to roll.
/ # cat test.rs
fn main() { println!("hello world"); }
/ # rustc test.rs
/ # ./test
hello world
# file test
test: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped
```
Fix generating rustc docs with non-default lib directory.
If `libdir` is set in `config.toml`, then the tool to generate the rustc docs was unable to run `rustc` because it could not find the shared libraries. The solution is to set the dylib search path to include the libdir.
I changed the API of `add_rustc_lib_path` to take `Command` instead of `Cargo` to try to share the code in several places. This is how it worked before https://github.com/rust-lang/rust/pull/64316, and I think this still retains the spirit of that change.
Fixes#76702
Split `core::slice` to smaller mods
Unfortunately the `#[lang = "slice"]` is too big (3003 lines), I cannot split it further.
Note for reviewer:
* I split to multiple commits for easier reviewing, but I could git squash them all to one if requested.
* Recommend pulling this change locally and using advanced git diff viewer or this command:
```
git show --reverse --color-moved=dimmed-zebra master..
```
---
I split core/slice/mod.rs to these modules:
* `ascii`: For operations on `[u8]`.
* `cmp`: For comparison operations on `[T]`, like PartialEq and SliceContains impl.
* `index`: For indexing operations like Index/IndexMut and SliceIndex.
* `iter`: For Iterator definitions and implementation on `[T]`.
- `macros`: For iterator! and forward_iterator! macros.
* `raw`: For free function to create `&[T]` or `&mut [T]` from pointer + length or a reference.
The heapsort wrapper in mod.rs is removed in favor of reexport from `sort::heapsort`.
Some errors, like if rustc is broken, or dylib search path is wrong,
will only display non-JSON errors. Show those, too, to make it easier to
debug a problem.
Ignore rustc_private items from std docs
By ignoring rustc_private items for non local impl block,
this may fix#74672 and fix#75588 .
This might suppress #76529 if it is simple enough for backport.
Attach tokens to all AST types used in `Nonterminal`
We perform token capturing when we have outer attributes (for nonterminals that support attributes - e.g. `Stmt`), or when we parse a `Nonterminal` for a `macro_rules!` argument. The full list of `Nonterminals` affected by this PR is:
* `NtBlock`
* `NtStmt`
* `NtTy`
* `NtMeta`
* `NtPath`
* `NtVis`
* `NtLiteral`
Of these nonterminals, only `NtStmt` and `NtLiteral` (which is actually just an `Expr`), support outer attributes - the rest only ever have token capturing perform when they match a `macro_rules!` argument.
This makes progress towards solving https://github.com/rust-lang/rust/issues/43081 - we now collect tokens for everything that might need them. However, we still need to handle `#[cfg]`, inner attributes, and misc pretty-printing issues (e.g. #75734)
I've separated the changes into (mostly) independent commits, which could be split into individual PRs for each `Nonterminal` variant. The purpose of having them all in one PR is to do a single Crater run for all of them.
Most of the changes in this PR are trivial (adding `tokens: None` everywhere we construct the various AST structs). The significant changes are:
* `ast::Visibility` is changed from `type Visibility = Spanned<VisibilityKind>` to a `struct Visibility { kind, span, tokens }`.
* `maybe_collect_tokens` is made generic, and used for both `ast::Expr` and `ast::Stmt`.
* Some of the statement-parsing functions are refactored so that we can capture the trailing semicolon.
* `Nonterminal` and `Expr` both grew by 8 bytes, as some of the structs which are stored inline (rather than behind a `P`) now have an `Option<TokenStream>` field. Hopefully the performance impact of doing this is negligible.
Add CONST_ITEM_MUTATION lint
Fixes#74053Fixes#55721
This PR adds a new lint `CONST_ITEM_MUTATION`.
Given an item `const FOO: SomeType = ..`, this lint fires on:
* Attempting to write directly to a field (`FOO.field = some_val`) or
array entry (`FOO.array_field[0] = val`)
* Taking a mutable reference to the `const` item (`&mut FOO`), including
through an autoderef `FOO.some_mut_self_method()`
The lint message explains that since each use of a constant creates a
new temporary, the original `const` item will not be modified.
We no longer lint assignments to const item fields in the
`temporary_assignment` lint, since this is now covered by the
`CONST_ITEM_MUTATION` lint.
Additionally, we `#![allow(const_item_mutation)]` in the
`borrow_interior_mutable_const.rs` test. Clippy UI tests are run with
`-D warnings`, which seems to cause builtin lints to prevent Clippy
lints from running.
Support dataflow problems on arbitrary lattices
This PR implements last of the proposed extensions I mentioned in the design meeting for the original dataflow refactor. It extends the current dataflow framework to work with arbitrary lattices, not just `BitSet`s. This is a prerequisite for dataflow-enabled MIR const-propagation. Personally, I am skeptical of the usefulness of doing const-propagation pre-monomorphization, since many useful constants only become known after monomorphization (e.g. `size_of::<T>()`) and users have a natural tendency to hand-optimize the rest. It's probably worth exprimenting with, however, and others have shown interest cc `@rust-lang/wg-mir-opt.`
The `Idx` associated type is moved from `AnalysisDomain` to `GenKillAnalysis` and replaced with an associated `Domain` type that must implement `JoinSemiLattice`. Like before, each `Analysis` defines the "bottom value" for its domain, but can no longer override the dataflow join operator. Analyses that want to use set intersection must now use the `lattice::Dual` newtype. `GenKillAnalysis` impls have an additional requirement that `Self::Domain: BorrowMut<BitSet<Self::Idx>>`, which effectively means that they must use `BitSet<Self::Idx>` or `lattice::Dual<BitSet<Self::Idx>>` as their domain.
Most of these changes were mechanical. However, because a `Domain` is no longer always a powerset of some index type, we can no longer use an `IndexVec<BasicBlock, GenKillSet<A::Idx>>>` to store cached block transfer functions. Instead, we use a boxed `dyn Fn` trait object. I discuss a few alternatives to the current approach in a commit message.
The majority of new lines of code are to preserve existing Graphviz diagrams for those unlucky enough to have to debug dataflow analyses. I find these diagrams incredibly useful when things are going wrong and considered regressing them unacceptable, especially the pretty-printing of `MovePathIndex`s, which are used in many dataflow analyses. This required a parallel `fmt` trait used only for printing dataflow domains, as well as a refactoring of the `graphviz` module now that we cannot expect the domain to be a `BitSet`. Some features did have to be removed, such as the gen/kill display mode (which I didn't use but existed to mirror the output of the old dataflow framework) and line wrapping. Since I had to rewrite much of it anyway, I took the opportunity to switch to a `Visitor` for printing dataflow state diffs instead of using cursors, which are error prone for code that must be generic over both forward and backward analyses. As a side-effect of this change, we no longer have quadratic behavior when writing graphviz diagrams for backward dataflow analyses.
r? `@pnkfelix`
debuginfo: Ignore HashMap .natvis tests before cdb 10.0.18362.1
CDB <10.0.18362.1 chokes on casts within HashMap's natvis visualizers. This PR adds support for "min-cdb-version" (per existing "min-gdb-version" and "min-lldb-version" filters) and uses it. CI uses a more recent version of CDB for testing and thus should still run the tests.
Credit to @petrochenkov per https://github.com/rust-lang/rust/issues/76352 for helping catch this.
### SDK Testing
| Win 10 SDK | x64 CDB | rustc 1.47.0-nightly (bf4342114 2020-08-25) built-in .natvis | Note |
| --------------------------------------------------------------------- | ----------------- | ------------------------------------------------------------- | ---- |
| [10.0.19041.0](https://go.microsoft.com/fwlink/p/?linkid=2120843) | 10.0.19041.1 | ✔️ | CI
| [10.0.18362.1](https://go.microsoft.com/fwlink/?linkid=2083338) | 10.0.18362.1 | ✔️ | MaulingMonkey
| [10.0.17763.0](https://go.microsoft.com/fwlink/p/?LinkID=2033908) | 10.0.17763.132 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.17134.12](https://go.microsoft.com/fwlink/p/?linkid=870807) | 10.0.17134.12 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.16299.91](https://go.microsoft.com/fwlink/p/?linkid=864422) | 10.0.16299.91 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.15063.468](https://go.microsoft.com/fwlink/p/?LinkId=845298) | 10.0.15063.468 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.14393.795](https://go.microsoft.com/fwlink/p/?LinkId=838916) | 10.0.14321.1024 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.` | petrochenkov
| [10.0.10586.212](https://go.microsoft.com/fwlink/p/?LinkID=698771) | 10.0.10586.567 | ❌ `Expected ')' at '+ 1)].__1'`
| [10.0.10240](https://go.microsoft.com/fwlink/p/?LinkId=619296) | 10.0.10240? | ❔ Untested
### Rust Testing
```cmd
x.py test --stage 1 src/tools/tidy
x.py test --stage 1 --build x86_64-pc-windows-msvc src\test\debuginfo
```
Also verified test still fails when intentionally broken w/ CDB version >= min-cdb-version.
Improve recovery on malformed format call
The token following a format expression should be a comma. However, when it is replaced with a similar token (such as a dot), then the corresponding error is emitted, but the token is treated as a comma, and the parsing step continues.
r? @petrochenkov
Update cargo
8 commits in 51b66125ba97d2906f461b3f4e0408f206299bb6..126907a7cfccbe93778530e6a6bbaa3adb6c515c
2020-08-19 20:22:52 +0000 to 2020-08-31 20:42:11 +0000
- Fix flakiness in close_output test (rust-lang/cargo#8668)
- Reload unstable table from config file in `reload_rooted_at` (rust-lang/cargo#8656)
- Bump to 0.49.0, update changelog (rust-lang/cargo#8659)
- Fix LTO with doctests. (rust-lang/cargo#8657)
- Add spaces after -C and -Z flags for consistency (rust-lang/cargo#8648)
- Fix cache_messages::rustdoc test broken on beta. (rust-lang/cargo#8653)
- fix: remove unnecessary allocations (rust-lang/cargo#8641)
- Fixed a spelling and some clippy warnings (rust-lang/cargo#8637)
Adds --bless support to test/run-make-fulldeps
The ability to "bless" output for some of these tests is critical to
making it practical to adapt tests to unrelated changes.
This is needed for new coverage tests, as shown in PR #76004 .
r? @tmandry
FYI: @wesleywiser
The ability to "bless" output for some of these tests is critical to
making it practical to adapt tests to unrelated changes.
This is needed for new coverage tests, as shown in PR #75828 (or its
derivative).
In 2e6f2e8855, we added a main function to the publish_toolstate.py script.
Unfortunately, we missed that the Python program implicitly declares global
variables in that code, which means that adding a function changes variable
scoping and breaks other code.
This commit avoids introducing that function and adds a warning to future
editors of the code.
Introduce expect snapshot testing library into rustc
Snapshot testing is a technique for writing maintainable unit tests.
Unlike usual `assert_eq!` tests, snapshot tests allow
to *automatically* upgrade expected values on test failure.
In a sense, snapshot tests are inline-version of our beloved
UI-tests.
Example:
A particular library we use, `expect_test` provides an `expect!`
macro, which creates a sort of self-updating string literal (by using
`file!` macro). Self-update is triggered by setting `UPDATE_EXPECT`
environmental variable (this info is printed during the test failure).
This library was extracted from rust-analyzer, where we use it for
most of our tests.
There are some other, more popular snapshot testing libraries:
* https://github.com/mitsuhiko/insta
* https://github.com/aaronabramov/k9
The main differences of `expect` are:
* first-class snapshot objects (so, tests can be written as functions,
rather than as macros)
* focus on inline-snapshots (but file snapshots are also supported)
* restricted feature set (only `assert_eq` and `assert_debug_eq`)
* no extra runtime (ie, no `cargo insta`)
See rust-analyzer/rust-analyzer#5101 for a
an extended comparison.
It is unclear if this testing style will stick with rustc in the long
run. At the moment, rustc is mainly tested via integrated UI tests.
But in the library-ified world, unit-tests will become somewhat more
important (that's why use use `rustc_lexer` library-ified library as
an example in this PR). Given that the cost of removal shouldn't be
too high, it probably makes sense to just see if this flies!
