It's described as a "backwards compatibility hack to keep the diff
small". Removing it requires only a modest amount of churn, and the
resulting code is clearer without the invisible derefs.
See also #135634, #149159, and rust-lang/hashbrown#662.
This includes an in-tree upgrade of `indexmap` as well, which uses the
new `HashTable` buckets API internally, hopefully impacting performance
for the better!
Move can be used only when both the compared operand and the operand on switch are move operands.
This commit directly changes to Copy, because I don't know if Move has beneficial.
The calls to `def_path_str` were outside the decorate callback in
`node_span_lint` which caused an ICE when the warning was an allowed
warning due to the call to `def_path_str` being executed but the
warning not actually being emitted.
JumpThreading: compute place and value indices on-demand
Profiling JumpThreading reveals that a large part of the runtime happens constructing the place and value `Map`. This is unfortunate, as jump-threading may end up not even doing anything.
The cause for this large up-front cost is following: `Map` attempts to create a `PlaceIndex` for each place that *may* hold a relevant value. This means all places that appear in MIR, but also all places whose value is accessed by a projection of a copy of a larger place.
This PR refactors the creation of `Map` to happen on-demand: place and value indices are created when threading computation happens.
The up-front mode is still relevant for DataflowConstProp, so is not touched.
Replace Rvalue::NullaryOp by a variant in mir::Operand.
Based on https://github.com/rust-lang/rust/pull/148151
This PR fully removes the MIR `Rvalue::NullaryOp`. After rust-lang/rust#148151, it was only useful for runtime checks like `ub_checks`, `contract_checks` and `overflow_checks`.
These are "runtime" checks, boolean constants that may only be `true` in codegen. It depends on a rustc flag passed to codegen, so we need to represent those flags cross-crate.
This PR replaces those runtime checks by special variants in MIR `ConstValue`. This allows code that expects constants to manipulate those as such, even if we may not always be able to evaluate them to actual scalars.
Introduces `BackendRepr::ScalableVector` corresponding to scalable
vector types annotated with `repr(scalable)` which lowers to a scalable
vector type in LLVM.
Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
misc coercion cleanups and handle safety correctly
r? lcnr
### "remove normalize call"
Fixesrust-lang/rust#132765
If the normalization fails we would sometimes get a `TypeError` containing inference variables created inside of the probe used by coercion. These would then get leaked out causing ICEs in diagnostics logic
### "leak check and lub for closure<->closure coerce-lubs of same defids"
Fixes https://github.com/rust-lang/trait-system-refactor-initiative/issues/233
```rust
fn peculiar() -> impl Fn(u8) -> u8 {
return |x| x + 1
}
```
the `|x| x + 1` expr has a type of `Closure(?31t)` which we wind up inferring the RPIT to. The `CoerceMany` `ret_coercion` for the whole `peculiar` typeck has an expected type of `RPIT` (unnormalized). When we type check the `return |x| x + 1` expr we go from the never type to `Closure(?31t)` which then participates in the `ret_coercion` giving us a `coerce-lub(RPIT, Closure(?31t))`.
Normalizing `RPIT` gives us some `Closure(?50t)` where `?31t` and `?50t` have been unified with `?31t` as the root var. `resolve_vars_if_possible` doesn't resolve infer vars to their roots so these wind up with different structural identities so the fast path doesn't apply and we fall back to coercing to a `fn` ptr. cc rust-lang/rust#147193 which also fixes this
New solver probably just gets more inference variables here because canonicalization + generally different approach to normalization of opaques. Idk :3
### FCP worthy stuffy
there are some other FCP worthy things but they're in my FCP comment which also contains some analysis of the breaking nature of the previously listed changes in this PR: https://github.com/rust-lang/rust/pull/148602#issuecomment-3503497467
Compute jump threading opportunities in a single pass
The current implementation of jump threading walks MIR CFG backwards from each `SwitchInt` terminator. This PR replaces this by a single postorder traversal of MIR. In theory, we could do a full fixpoint dataflow analysis, but this has low returns as we forbid threading through a loop header.
The second commit in this PR modifies the carried state to a lighter data structure. The current implementation uses some kind of `IndexVec<ValueIndex, &[Condition]>`. This is needlessly heavy, as the state rarely ever carries more than a few `Condition`s. The first commit replaces this state with a simpler `&[Condition]`, and puts the corresponding `ValueIndex` inside `Condition`.
The three later commits are perf tweaks.
The sixth commit is the main change. Instead of carrying the goto target inside the condition, we maintain a set of conditions associated with each block, and their consequences in following blocks. Think: if this condition is fulfilled in this block, then that condition is fulfilled in that block. This makes the threading algorithm much easier to implement, without the extra bookkeeping of `ThreadingOpportunity` we had.
Later commits modify that algorithm to shrink the set of duplicated blocks. By propagating fulfilled conditions down the CFG, and trimming costly threads.
Fix MaybeUninit codegen using GVN
This is an alternative to https://github.com/rust-lang/rust/pull/142837, based on https://github.com/rust-lang/rust/pull/146355#discussion_r2421651968.
The general approach I took here is to aggressively propagate anything that is entirely uninitialized. GVN generally takes the approach of only synthesizing small types, but we need to generate large consts to fix the codegen issue.
I also added a special case to MIR dumps for this where now an entirely uninit const is printed as `const <uninit>`, because otherwise we end up with extremely verbose dumps of the new consts.
After GVN though, we still end up with a lot of MIR that looks like this:
```
StorageLive(_1);
_1 = const <uninit>;
_2 = &raw mut _1;
```
Which will break tests/codegen-llvm/maybeuninit-rvo.rs with the naive lowering. I think the ideal fix here is to somehow omit these `_1 = const <uninit>` assignments that come directly after a StorageLive, but I'm not sure how to do that. For now at least, ignoring such assignments (even if they don't come right after a StorageLive) in codegen seems to work.
Note that since GVN is based on synthesizing a `ConstValue` which has a defined layout, this scenario still gets deoptimized by LLVM.
```rust
#![feature(rustc_attrs)]
#![crate_type = "lib"]
use std::mem::MaybeUninit;
#[unsafe(no_mangle)]
pub fn oof() -> [[MaybeUninit<u8>; 8]; 8] {
#[rustc_no_mir_inline]
pub fn inner<T: Copy>() -> [[MaybeUninit<T>; 8]; 8] {
[[MaybeUninit::uninit(); 8]; 8]
}
inner()
}
```
This case can be handled correctly if enough inlining has happened, or it could be handled by post-mono GVN. Synthesizing `UnevaluatedConst` or some other special kind of const seems dubious.
Turn moves into copies after copy propagation
Previously copy propagation presumed that there is further unspecified distinction between move operands and copy operands in assignments and propagated moves from assignments into terminators. This is inconsistent with current operational semantics.
Turn moves into copies after copy propagation to preserve existing behavior.
Fixes https://github.com/rust-lang/rust/issues/137936.
Fixes https://github.com/rust-lang/rust/issues/146423.
r? `@cjgillot`
Skip unused variables warning for unreachable code
Fixesrust-lang/rust#148373
These warnings are not reported on stable branch, but are now reported on the beta.
I tried another solution to record whether a `local` is reachable in `find_dead_assignments`, but the code in this PR seems simpler.
r? `@cjgillot`
Fix the issue of unused assignment from MIR liveness checking
Fixesrust-lang/rust#148960Fixesrust-lang/rust#148418
r? ``@davidtwco``
cc ``@cjgillot``
My first try on MIR related code, so it may not be the best fix.
Allow unnormalized types in drop elaboration
I work on a [rustc driver](https://github.com/AeneasVerif/charon) that aims to extract the full polymorphic MIR of a crate. Currently the one thing I can't get is drop glue because it fails on unnormalizable types like the fields of `Cow` or `NonZero`.
This PR removes the one check for unnormalized types in drop elaboration. It's a `span_delay_bug` so only there to help catch mistakes. I think that's fine to remove? If something downstream relies on types being normalized the right approach is for MIR validation to check for normalized types, not that one random check.
autodiff rlib handling
As I learned recently, we now apparently support rlib builds already in some cases.
With the last hint from saethlin this seems to now cover all cases. To be sure I'll add a few more testcases before I mark it as ready.
Once this PR lands, we should to the best of my knowledge, support autodiff in almost code locations, only vtable/dyn ptr remain unsupported for now.
r? ghost
closes: https://github.com/rust-lang/rust/issues/148856
closes: https://github.com/rust-lang/rust/issues/137520
