Store the type of each GVN value
MIR is fully typed, so type information is an integral part of what defines a value. GVN currently tries to circumvent storing types, which creates all sorts of complexities.
This PR stores the type along with the enum `Value` when defining a value index. This allows to simplify a lot of code.
Fixesrust-lang/rust#128094Fixesrust-lang/rust#135128
r? ``````@ghost`````` for perf
Only inherit local hash for paths
`DefPathHash`, as the counterpart of `DefId` that is stable across compiler invocations, is comprised of 2 parts. The first one is the `StableCrateId`, stable form of `CrateNum`. The second is 64 complementary bits to identify the crate-local definition.
The current implementation always hashes the full 128 bits when (1) trying to create a new child `DefPathHash` or (2) hashing a `CrateNum` or a `LocalDefId`. But we only need half that information: `LocalDefId` means that the `StableCrateId` is always the current crate's ; `CrateNum` means that we do not care about the local part.
As stable hashing is very hot in the query system, in particular hashing definitions, this is a big deal.
We still want the local part to change when the `StableCrateId` changes, to make incr-compilation errors less painful, ie. increase the likelihood that if will magically disappear by changing some code.
This PR sprinkles some `#[inline]` attributes on small functions that appeared in profiles.
Apply effects to `otherwise` edge in dataflow analysis
This allows `ElaborateDrops` to remove drops when a `match` wildcard arm covers multiple no-Drop enum variants. It modifies dataflow analysis to update the `MaybeUninitializedPlaces` and `MaybeInitializedPlaces` data for a block reached through an `otherwise` edge.
Fixesrust-lang/rust#142705.
Dont resolve instance of root in `mir_callgraph_cyclic`
`Instance::try_resolve` on a default trait body method will always fail, since it's still possible to further substitute. This leads to a cycle, since in `tests/mir-opt/inline_default_trait_body.rs`, both `Trait::a` and `Trait::b` need to consider the other to be cyclical, but since we couldn't resolve a body, we'd just consider *nothing* to be cyclical.
The root instance we care about when computing `mir_callgraph_cyclic` is trivial to compute (it's just `InstanceKind::Item`), so just replace it with a call to `Instance::new_raw`.
r? `@cjgillot` `@oli-obk`
Fixesrust-lang/rust#143534
Reason about borrowed classes in CopyProp.
Fixes https://github.com/rust-lang/rust/issues/141122
The current implementation of `CopyProp` avoids unifying two borrowed locals, as this would change the result of address comparison.
However, the implementation was inconsistent with the general algorithm, which identifies equivalence classes of locals and then replaces all locals by a single representative of their equivalence class.
This PR fixes it by forbidding the unification of two *classes* if any of those contain a borrowed local.
mir-opt: Do not create storage marks in EarlyOtherwiseBranch
Fixes#141212.
The first commit add `StorageDead` by creating new indirect BB that makes CFG more complicated, but I think it's better to just not create storage marks.
r? mir-opt
`slice.get(i)` should use a slice projection in MIR, like `slice[i]` does
`slice[i]` is built-in magic, so ends up being quite different from `slice.get(i)` in MIR, even though they're both doing nearly identical operations -- checking the length of the slice then getting a ref/ptr to the element if it's in-bounds.
This PR adds a `slice_get_unchecked` intrinsic for `impl SliceIndex for usize` to use to fix that, so it no longer needs to do a bunch of lines of pointer math and instead just gets the obvious single statement. (This is *not* used for the range versions, since `slice[i..]` and `slice[..k]` can't use the mir Slice projection as they're using fenceposts, not indices.)
I originally tried to do this with some kind of GVN pattern, but realized that I'm pretty sure it's not legal to optimize `BinOp::Offset` to `PlaceElem::Index` without an extremely complicated condition. Basically, the problem is that the `Index` projection on a dereferenced slice pointer *cares about the metadata*, since it's UB to `PlaceElem::Index` outside the range described by the metadata. But then you cast the fat pointer to a thin pointer then offset it, that *ignores* the slice length metadata, so it's possible to write things that are legal with `Offset` but would be UB if translated in the obvious way to `Index`. Checking (or even determining) the necessary conditions for that would be complicated and error-prone, whereas this intrinsic-based approach is quite straight-forward.
Zero backend changes, because it just lowers to MIR, so it's already supported naturally by CTFE/Miri/cg_llvm/cg_clif.
Add fast path for maybe-initializedness in liveness
r? `@matthewjasper`
Correct me if I'm wrong Matthew, but my understanding is that
1. `MaybeInitializedPlaces` is currently eagerly computed, in `do_mir_borrowck`
2. but this data is only used in liveness
3. and `liveness::trace` actually only uses it for drop-liveness
This PR moves the computation to `liveness::trace` which looks to be its only use-site. We also add a fast path there, so that it's only computed by drop-liveness.
This is interesting because 1) liveness is only computed for relevant live locals, 2) drop-liveness is only computed for relevant live locals with >0 drop points; 0 is the common case from our benchmarks, as far as I can tell, so even just computing the entire data lazily helps.
It seems possible to also reduce the domain here, and speed up the analysis for the cases where it has to be computed -- so I've left a fixme for that, and may look into it soon.
(I've come upon this while doing implementation work for polonius, so don't be too enamored with possible wins: the goal is to reduce the eventual polonius overhead and make it more palatable 😓)
gvn: bail out unavoidable non-ssa locals in repeat
Fixes#141251.
We cannot transform `*elem` to `array[idx1]` in the following code, as `idx1` has already been modified.
```rust
mir! {
let array;
let elem;
{
array = [*val; 5];
elem = &array[idx1];
idx1 = idx2;
RET = *elem;
Return()
}
}
```
Perhaps I could transform it to `array[0]`, but I prefer the conservative approach.
r? mir-opt
Add some track_caller info to precondition panics
Currently, when you encounter a precondition check, you'll always get the caller location of the implementation of the precondition checks. But with this PR, you'll be told the location of the invalid call. Which is useful.
I thought of this while looking at https://github.com/rust-lang/rust/pull/129642#issuecomment-2311703898.
The changes to `tests/ui/const*` happen because the const-eval interpreter skips `#[track_caller]` frames in its backtraces.
The perf implications of this are:
* Increased debug binary sizes. The caller_location implementation requires that the additional data we want to display here be stored in const allocations, which are deduplicated but not across crates. There is no impact on optimized build sizes. The panic path and the caller location data get optimized out.
* The compile time hit to opt-incr-patched bitmaps happens because the patch changes the line number of some function calls with precondition checks, causing us to go from 0 dirty CGUs to 1 dirty CGU.
* The other compile time hits are marginal but real, and due to doing a handful of new queries. Adding more useful data isn't completely free.
We cannot transform `*elem` to `array[idx1]` in the following code,
as `idx1` has already been modified.
```rust
mir! {
let array;
let elem;
{
array = [*val; 5];
elem = &array[idx1];
idx1 = idx2;
RET = *elem;
Return()
}
}
```
coverage: Detect unused local file IDs to avoid an LLVM assertion
Each function's coverage metadata contains a *local file table* that maps local file IDs (used by the function's mapping regions) to global file IDs (shared by all functions in the same CGU).
LLVM requires all local file IDs to have at least one mapping region, and has an assertion that will fail if it detects a local file ID with no regions. To make sure that assertion doesn't fire, we need to detect and skip functions whose metadata would trigger it.
(This can't actually happen yet, because currently all of a function's spans must belong to the same file and expansion. But this will be an important edge case when adding expansion region support.)
remove intrinsics::drop_in_place
This was only ever accidentally stable, and has been marked as deprecated since Rust 1.52, released almost 4 years ago. We've removed the old serialization `derive`s, maybe we can remove this one as well?
As suggested by ``@jhpratt,`` let's see what crater says for this one.
mir-opt: execute MatchBranchSimplification after GVN
This can provide more opportunities for MatchBranchSimplification.
Currently, rustc does not optimize the following code into a single statement at mir-opt, and this PR fixes the first case.
```rust
pub fn match1(c: bool, v1: i32, v2: i32) -> i32 {
if c { v1 - v2 } else { v1 - v2 }
}
pub fn match2(c: bool, v1: i32) -> i32 {
if c { v1 - 1 } else { v1 - 1 }
}
```
https://rust.godbolt.org/z/Y8xPMjrfM
r? mir-opt
