Support dataflow problems on arbitrary lattices
This PR implements last of the proposed extensions I mentioned in the design meeting for the original dataflow refactor. It extends the current dataflow framework to work with arbitrary lattices, not just `BitSet`s. This is a prerequisite for dataflow-enabled MIR const-propagation. Personally, I am skeptical of the usefulness of doing const-propagation pre-monomorphization, since many useful constants only become known after monomorphization (e.g. `size_of::<T>()`) and users have a natural tendency to hand-optimize the rest. It's probably worth exprimenting with, however, and others have shown interest cc `@rust-lang/wg-mir-opt.`
The `Idx` associated type is moved from `AnalysisDomain` to `GenKillAnalysis` and replaced with an associated `Domain` type that must implement `JoinSemiLattice`. Like before, each `Analysis` defines the "bottom value" for its domain, but can no longer override the dataflow join operator. Analyses that want to use set intersection must now use the `lattice::Dual` newtype. `GenKillAnalysis` impls have an additional requirement that `Self::Domain: BorrowMut<BitSet<Self::Idx>>`, which effectively means that they must use `BitSet<Self::Idx>` or `lattice::Dual<BitSet<Self::Idx>>` as their domain.
Most of these changes were mechanical. However, because a `Domain` is no longer always a powerset of some index type, we can no longer use an `IndexVec<BasicBlock, GenKillSet<A::Idx>>>` to store cached block transfer functions. Instead, we use a boxed `dyn Fn` trait object. I discuss a few alternatives to the current approach in a commit message.
The majority of new lines of code are to preserve existing Graphviz diagrams for those unlucky enough to have to debug dataflow analyses. I find these diagrams incredibly useful when things are going wrong and considered regressing them unacceptable, especially the pretty-printing of `MovePathIndex`s, which are used in many dataflow analyses. This required a parallel `fmt` trait used only for printing dataflow domains, as well as a refactoring of the `graphviz` module now that we cannot expect the domain to be a `BitSet`. Some features did have to be removed, such as the gen/kill display mode (which I didn't use but existed to mirror the output of the old dataflow framework) and line wrapping. Since I had to rewrite much of it anyway, I took the opportunity to switch to a `Visitor` for printing dataflow state diffs instead of using cursors, which are error prone for code that must be generic over both forward and backward analyses. As a side-effect of this change, we no longer have quadratic behavior when writing graphviz diagrams for backward dataflow analyses.
r? `@pnkfelix`
debuginfo: Ignore HashMap .natvis tests before cdb 10.0.18362.1
CDB <10.0.18362.1 chokes on casts within HashMap's natvis visualizers. This PR adds support for "min-cdb-version" (per existing "min-gdb-version" and "min-lldb-version" filters) and uses it. CI uses a more recent version of CDB for testing and thus should still run the tests.
Credit to @petrochenkov per https://github.com/rust-lang/rust/issues/76352 for helping catch this.
### SDK Testing
| Win 10 SDK | x64 CDB | rustc 1.47.0-nightly (bf4342114 2020-08-25) built-in .natvis | Note |
| --------------------------------------------------------------------- | ----------------- | ------------------------------------------------------------- | ---- |
| [10.0.19041.0](https://go.microsoft.com/fwlink/p/?linkid=2120843) | 10.0.19041.1 | ✔️ | CI
| [10.0.18362.1](https://go.microsoft.com/fwlink/?linkid=2083338) | 10.0.18362.1 | ✔️ | MaulingMonkey
| [10.0.17763.0](https://go.microsoft.com/fwlink/p/?LinkID=2033908) | 10.0.17763.132 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.17134.12](https://go.microsoft.com/fwlink/p/?linkid=870807) | 10.0.17134.12 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.16299.91](https://go.microsoft.com/fwlink/p/?linkid=864422) | 10.0.16299.91 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.15063.468](https://go.microsoft.com/fwlink/p/?LinkId=845298) | 10.0.15063.468 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.`
| [10.0.14393.795](https://go.microsoft.com/fwlink/p/?LinkId=838916) | 10.0.14321.1024 | ❌ `Unable to find type 'tuple<u64,u64> *' for cast.` | petrochenkov
| [10.0.10586.212](https://go.microsoft.com/fwlink/p/?LinkID=698771) | 10.0.10586.567 | ❌ `Expected ')' at '+ 1)].__1'`
| [10.0.10240](https://go.microsoft.com/fwlink/p/?LinkId=619296) | 10.0.10240? | ❔ Untested
### Rust Testing
```cmd
x.py test --stage 1 src/tools/tidy
x.py test --stage 1 --build x86_64-pc-windows-msvc src\test\debuginfo
```
Also verified test still fails when intentionally broken w/ CDB version >= min-cdb-version.
Improve recovery on malformed format call
The token following a format expression should be a comma. However, when it is replaced with a similar token (such as a dot), then the corresponding error is emitted, but the token is treated as a comma, and the parsing step continues.
r? @petrochenkov
Update cargo
8 commits in 51b66125ba97d2906f461b3f4e0408f206299bb6..126907a7cfccbe93778530e6a6bbaa3adb6c515c
2020-08-19 20:22:52 +0000 to 2020-08-31 20:42:11 +0000
- Fix flakiness in close_output test (rust-lang/cargo#8668)
- Reload unstable table from config file in `reload_rooted_at` (rust-lang/cargo#8656)
- Bump to 0.49.0, update changelog (rust-lang/cargo#8659)
- Fix LTO with doctests. (rust-lang/cargo#8657)
- Add spaces after -C and -Z flags for consistency (rust-lang/cargo#8648)
- Fix cache_messages::rustdoc test broken on beta. (rust-lang/cargo#8653)
- fix: remove unnecessary allocations (rust-lang/cargo#8641)
- Fixed a spelling and some clippy warnings (rust-lang/cargo#8637)
Adds --bless support to test/run-make-fulldeps
The ability to "bless" output for some of these tests is critical to
making it practical to adapt tests to unrelated changes.
This is needed for new coverage tests, as shown in PR #76004 .
r? @tmandry
FYI: @wesleywiser
The ability to "bless" output for some of these tests is critical to
making it practical to adapt tests to unrelated changes.
This is needed for new coverage tests, as shown in PR #75828 (or its
derivative).
In 2e6f2e8855, we added a main function to the publish_toolstate.py script.
Unfortunately, we missed that the Python program implicitly declares global
variables in that code, which means that adding a function changes variable
scoping and breaks other code.
This commit avoids introducing that function and adds a warning to future
editors of the code.
Introduce expect snapshot testing library into rustc
Snapshot testing is a technique for writing maintainable unit tests.
Unlike usual `assert_eq!` tests, snapshot tests allow
to *automatically* upgrade expected values on test failure.
In a sense, snapshot tests are inline-version of our beloved
UI-tests.
Example:
A particular library we use, `expect_test` provides an `expect!`
macro, which creates a sort of self-updating string literal (by using
`file!` macro). Self-update is triggered by setting `UPDATE_EXPECT`
environmental variable (this info is printed during the test failure).
This library was extracted from rust-analyzer, where we use it for
most of our tests.
There are some other, more popular snapshot testing libraries:
* https://github.com/mitsuhiko/insta
* https://github.com/aaronabramov/k9
The main differences of `expect` are:
* first-class snapshot objects (so, tests can be written as functions,
rather than as macros)
* focus on inline-snapshots (but file snapshots are also supported)
* restricted feature set (only `assert_eq` and `assert_debug_eq`)
* no extra runtime (ie, no `cargo insta`)
See rust-analyzer/rust-analyzer#5101 for a
an extended comparison.
It is unclear if this testing style will stick with rustc in the long
run. At the moment, rustc is mainly tested via integrated UI tests.
But in the library-ified world, unit-tests will become somewhat more
important (that's why use use `rustc_lexer` library-ified library as
an example in this PR). Given that the cost of removal shouldn't be
too high, it probably makes sense to just see if this flies!
Snapshot testing is a technique for writing maintainable unit tests.
Unlike usual `assert_eq!` tests, snapshot tests allow
to *automatically* upgrade expected values on test failure.
In a sense, snapshot tests are inline-version of our beloved
UI-tests.
Example:
A particular library we use, `expect_test` provides an `expect!`
macro, which creates a sort of self-updating string literal (by using
`file!` macro). Self-update is triggered by setting `UPDATE_EXPECT`
environmental variable (this info is printed during the test failure).
This library was extracted from rust-analyzer, where we use it for
most of our tests.
There are some other, more popular snapshot testing libraries:
* https://github.com/mitsuhiko/insta
* https://github.com/aaronabramov/k9
The main differences of `expect` are:
* first-class snapshot objects (so, tests can be written as functions,
rather than as macros)
* focus on inline-snapshots (but file snapshots are also supported)
* restricted feature set (only `assert_eq` and `assert_debug_eq`)
* no extra runtime (ie, no `cargo insta`)
See https://github.com/rust-analyzer/rust-analyzer/pull/5101 for a
an extended comparison.
It is unclear if this testing style will stick with rustc in the long
run. At the moment, rustc is mainly tested via integrated UI tests.
But in the library-ified world, unit-tests will become somewhat more
important (that's why use use `rustc_lexer` library-ified library as
an example in this PR). Given that the cost of removal shouldn't be
too high, it probably makes sense to just see if this flies!
Fixes clippy::redundant_static_lifetimes and clippy::toplevel_ref_arg
I also replaced some .expect("") calls with .unwrap()s since there was no message passed by the .expect() anyway.
Capture tokens for Pat used in macro_rules! argument
This extends PR #73293 to handle patterns (Pat). Unlike expressions,
patterns do not support custom attributes, so we only need to capture
tokens during macro_rules! argument parsing.
This extends PR #73293 to handle patterns (Pat). Unlike expressions,
patterns do not support custom attributes, so we only need to capture
tokens during macro_rules! argument parsing.
