da630ac79d
indirect immutable freeze by-value function parameters.
Right now, `rustc` only examines function signatures and the platform ABI when
determining the LLVM attributes to apply to parameters. This results in missed
optimizations, because there are some attributes that can be determined via
analysis of the MIR making up the function body. In particular, `readonly`
could be applied to most indirectly-passed by-value function arguments
(specifically, those that are freeze and are observed not to be mutated), but
it currently is not.
This patch introduces the machinery that allows `rustc` to determine those
attributes. It consists of a query, `deduced_param_attrs`, that, when
evaluated, analyzes the MIR of the function to determine supplementary
attributes. The results of this query for each function are written into the
crate metadata so that the deduced parameter attributes can be applied to
cross-crate functions. In this patch, we simply check the parameter for
mutations to determine whether the `readonly` attribute should be applied to
parameters that are indirect immutable freeze by-value. More attributes could
conceivably be deduced in the future: `nocapture` and `noalias` come to mind.
Adding `readonly` to indirect function parameters where applicable enables some
potential optimizations in LLVM that are discussed in [issue 103103] and [PR
103070] around avoiding stack-to-stack memory copies that appear in functions
like `core::fmt::Write::write_fmt` and `core::panicking::assert_failed`. These
functions pass a large structure unchanged by value to a subfunction that also
doesn't mutate it. Since the structure in this case is passed as an indirect
parameter, it's a pointer from LLVM's perspective. As a result, the
intermediate copy of the structure that our codegen emits could be optimized
away by LLVM's MemCpyOptimizer if it knew that the pointer is `readonly
nocapture noalias` in both the caller and callee. We already pass `nocapture
noalias`, but we're missing `readonly`, as we can't determine whether a
by-value parameter is mutated by examining the signature in Rust. I didn't have
much success with having LLVM infer the `readonly` attribute, even with fat
LTO; it seems that deducing it at the MIR level is necessary.
No large benefits should be expected from this optimization *now*; LLVM needs
some changes (discussed in [PR 103070]) to more aggressively use the `noalias
nocapture readonly` combination in its alias analysis. I have some LLVM patches
for these optimizations and have had them looked over. With all the patches
applied locally, I enabled LLVM to remove all the `memcpy`s from the following
code:
```rust
fn main() {
println!("Hello {}", 3);
}
```
which is a significant codegen improvement over the status quo. I expect that
if this optimization kicks in in multiple places even for such a simple
program, then it will apply to Rust code all over the place.
[issue 103103]: https://github.com/rust-lang/rust/issues/103103
[PR 103070]: https://github.com/rust-lang/rust/pull/103070
235 lines
6.2 KiB
Rust
235 lines
6.2 KiB
Rust
// compile-flags: -O -C no-prepopulate-passes
|
|
|
|
#![crate_type = "lib"]
|
|
#![feature(rustc_attrs)]
|
|
|
|
use std::mem::MaybeUninit;
|
|
use std::num::NonZeroU64;
|
|
use std::marker::PhantomPinned;
|
|
|
|
pub struct S {
|
|
_field: [i32; 8],
|
|
}
|
|
|
|
pub struct UnsafeInner {
|
|
_field: std::cell::UnsafeCell<i16>,
|
|
}
|
|
|
|
pub struct NotUnpin {
|
|
_field: i32,
|
|
_marker: PhantomPinned,
|
|
}
|
|
|
|
pub enum MyBool {
|
|
True,
|
|
False,
|
|
}
|
|
|
|
// CHECK: noundef zeroext i1 @boolean(i1 noundef zeroext %x)
|
|
#[no_mangle]
|
|
pub fn boolean(x: bool) -> bool {
|
|
x
|
|
}
|
|
|
|
// CHECK: i8 @maybeuninit_boolean(i8 %x)
|
|
#[no_mangle]
|
|
pub fn maybeuninit_boolean(x: MaybeUninit<bool>) -> MaybeUninit<bool> {
|
|
x
|
|
}
|
|
|
|
// CHECK: noundef zeroext i1 @enum_bool(i1 noundef zeroext %x)
|
|
#[no_mangle]
|
|
pub fn enum_bool(x: MyBool) -> MyBool {
|
|
x
|
|
}
|
|
|
|
// CHECK: i8 @maybeuninit_enum_bool(i8 %x)
|
|
#[no_mangle]
|
|
pub fn maybeuninit_enum_bool(x: MaybeUninit<MyBool>) -> MaybeUninit<MyBool> {
|
|
x
|
|
}
|
|
|
|
// CHECK: noundef i32 @char(i32 noundef %x)
|
|
#[no_mangle]
|
|
pub fn char(x: char) -> char {
|
|
x
|
|
}
|
|
|
|
// CHECK: i32 @maybeuninit_char(i32 %x)
|
|
#[no_mangle]
|
|
pub fn maybeuninit_char(x: MaybeUninit<char>) -> MaybeUninit<char> {
|
|
x
|
|
}
|
|
|
|
// CHECK: i64 @int(i64 %x)
|
|
#[no_mangle]
|
|
pub fn int(x: u64) -> u64 {
|
|
x
|
|
}
|
|
|
|
// CHECK: noundef i64 @nonzero_int(i64 noundef %x)
|
|
#[no_mangle]
|
|
pub fn nonzero_int(x: NonZeroU64) -> NonZeroU64 {
|
|
x
|
|
}
|
|
|
|
// CHECK: i64 @option_nonzero_int(i64 %x)
|
|
#[no_mangle]
|
|
pub fn option_nonzero_int(x: Option<NonZeroU64>) -> Option<NonZeroU64> {
|
|
x
|
|
}
|
|
|
|
// CHECK: @readonly_borrow({{i32\*|ptr}} noalias noundef readonly align 4 dereferenceable(4) %_1)
|
|
// FIXME #25759 This should also have `nocapture`
|
|
#[no_mangle]
|
|
pub fn readonly_borrow(_: &i32) {
|
|
}
|
|
|
|
// CHECK: @static_borrow({{i32\*|ptr}} noalias noundef readonly align 4 dereferenceable(4) %_1)
|
|
// static borrow may be captured
|
|
#[no_mangle]
|
|
pub fn static_borrow(_: &'static i32) {
|
|
}
|
|
|
|
// CHECK: @named_borrow({{i32\*|ptr}} noalias noundef readonly align 4 dereferenceable(4) %_1)
|
|
// borrow with named lifetime may be captured
|
|
#[no_mangle]
|
|
pub fn named_borrow<'r>(_: &'r i32) {
|
|
}
|
|
|
|
// CHECK: @unsafe_borrow({{i16\*|ptr}} noundef nonnull align 2 %_1)
|
|
// unsafe interior means this isn't actually readonly and there may be aliases ...
|
|
#[no_mangle]
|
|
pub fn unsafe_borrow(_: &UnsafeInner) {
|
|
}
|
|
|
|
// CHECK: @mutable_unsafe_borrow({{i16\*|ptr}} noalias noundef align 2 dereferenceable(2) %_1)
|
|
// ... unless this is a mutable borrow, those never alias
|
|
#[no_mangle]
|
|
pub fn mutable_unsafe_borrow(_: &mut UnsafeInner) {
|
|
}
|
|
|
|
// CHECK: @mutable_borrow({{i32\*|ptr}} noalias noundef align 4 dereferenceable(4) %_1)
|
|
// FIXME #25759 This should also have `nocapture`
|
|
#[no_mangle]
|
|
pub fn mutable_borrow(_: &mut i32) {
|
|
}
|
|
|
|
#[no_mangle]
|
|
// CHECK: @mutable_notunpin_borrow({{i32\*|ptr}} noundef align 4 dereferenceable(4) %_1)
|
|
// This one is *not* `noalias` because it might be self-referential.
|
|
pub fn mutable_notunpin_borrow(_: &mut NotUnpin) {
|
|
}
|
|
|
|
// CHECK: @notunpin_borrow({{i32\*|ptr}} noalias noundef readonly align 4 dereferenceable(4) %_1)
|
|
// But `&NotUnpin` behaves perfectly normal.
|
|
#[no_mangle]
|
|
pub fn notunpin_borrow(_: &NotUnpin) {
|
|
}
|
|
|
|
// CHECK: @indirect_struct({{%S\*|ptr}} noalias nocapture noundef readonly dereferenceable(32) %_1)
|
|
#[no_mangle]
|
|
pub fn indirect_struct(_: S) {
|
|
}
|
|
|
|
// CHECK: @borrowed_struct({{%S\*|ptr}} noalias noundef readonly align 4 dereferenceable(32) %_1)
|
|
// FIXME #25759 This should also have `nocapture`
|
|
#[no_mangle]
|
|
pub fn borrowed_struct(_: &S) {
|
|
}
|
|
|
|
// CHECK: @raw_struct({{%S\*|ptr}} %_1)
|
|
#[no_mangle]
|
|
pub fn raw_struct(_: *const S) {
|
|
}
|
|
|
|
// `Box` can get deallocated during execution of the function, so it should
|
|
// not get `dereferenceable`.
|
|
// CHECK: noalias noundef nonnull align 4 {{i32\*|ptr}} @_box({{i32\*|ptr}} noalias noundef nonnull align 4 %x)
|
|
#[no_mangle]
|
|
pub fn _box(x: Box<i32>) -> Box<i32> {
|
|
x
|
|
}
|
|
|
|
// CHECK: @struct_return({{%S\*|ptr}} noalias nocapture noundef sret(%S) dereferenceable(32){{( %0)?}})
|
|
#[no_mangle]
|
|
pub fn struct_return() -> S {
|
|
S {
|
|
_field: [0, 0, 0, 0, 0, 0, 0, 0]
|
|
}
|
|
}
|
|
|
|
// Hack to get the correct size for the length part in slices
|
|
// CHECK: @helper([[USIZE:i[0-9]+]] %_1)
|
|
#[no_mangle]
|
|
pub fn helper(_: usize) {
|
|
}
|
|
|
|
// CHECK: @slice({{\[0 x i8\]\*|ptr}} noalias noundef nonnull readonly align 1 %_1.0, [[USIZE]] %_1.1)
|
|
// FIXME #25759 This should also have `nocapture`
|
|
#[no_mangle]
|
|
pub fn slice(_: &[u8]) {
|
|
}
|
|
|
|
// CHECK: @mutable_slice({{\[0 x i8\]\*|ptr}} noalias noundef nonnull align 1 %_1.0, [[USIZE]] %_1.1)
|
|
// FIXME #25759 This should also have `nocapture`
|
|
#[no_mangle]
|
|
pub fn mutable_slice(_: &mut [u8]) {
|
|
}
|
|
|
|
// CHECK: @unsafe_slice({{\[0 x i16\]\*|ptr}} noundef nonnull align 2 %_1.0, [[USIZE]] %_1.1)
|
|
// unsafe interior means this isn't actually readonly and there may be aliases ...
|
|
#[no_mangle]
|
|
pub fn unsafe_slice(_: &[UnsafeInner]) {
|
|
}
|
|
|
|
// CHECK: @raw_slice({{\[0 x i8\]\*|ptr}} %_1.0, [[USIZE]] %_1.1)
|
|
#[no_mangle]
|
|
pub fn raw_slice(_: *const [u8]) {
|
|
}
|
|
|
|
// CHECK: @str({{\[0 x i8\]\*|ptr}} noalias noundef nonnull readonly align 1 %_1.0, [[USIZE]] %_1.1)
|
|
// FIXME #25759 This should also have `nocapture`
|
|
#[no_mangle]
|
|
pub fn str(_: &[u8]) {
|
|
}
|
|
|
|
// CHECK: @trait_borrow({{\{\}\*|ptr}} noundef nonnull align 1 %_1.0, {{.+}} noalias noundef readonly align {{.*}} dereferenceable({{.*}}) %_1.1)
|
|
// FIXME #25759 This should also have `nocapture`
|
|
#[no_mangle]
|
|
pub fn trait_borrow(_: &Drop) {
|
|
}
|
|
|
|
// CHECK: @trait_raw({{\{\}\*|ptr}} %_1.0, {{.+}} noalias noundef readonly align {{.*}} dereferenceable({{.*}}) %_1.1)
|
|
#[no_mangle]
|
|
pub fn trait_raw(_: *const Drop) {
|
|
}
|
|
|
|
// CHECK: @trait_box({{\{\}\*|ptr}} noalias noundef nonnull align 1{{( %0)?}}, {{.+}} noalias noundef readonly align {{.*}} dereferenceable({{.*}}){{( %1)?}})
|
|
#[no_mangle]
|
|
pub fn trait_box(_: Box<Drop>) {
|
|
}
|
|
|
|
// CHECK: { {{i8\*|ptr}}, {{i8\*|ptr}} } @trait_option({{i8\*|ptr}} noalias noundef align 1 %x.0, {{i8\*|ptr}} %x.1)
|
|
#[no_mangle]
|
|
pub fn trait_option(x: Option<Box<Drop>>) -> Option<Box<Drop>> {
|
|
x
|
|
}
|
|
|
|
// CHECK: { {{\[0 x i16\]\*|ptr}}, [[USIZE]] } @return_slice({{\[0 x i16\]\*|ptr}} noalias noundef nonnull readonly align 2 %x.0, [[USIZE]] %x.1)
|
|
#[no_mangle]
|
|
pub fn return_slice(x: &[u16]) -> &[u16] {
|
|
x
|
|
}
|
|
|
|
// CHECK: { i16, i16 } @enum_id_1(i16 noundef %x.0, i16 %x.1)
|
|
#[no_mangle]
|
|
pub fn enum_id_1(x: Option<Result<u16, u16>>) -> Option<Result<u16, u16>> {
|
|
x
|
|
}
|
|
|
|
// CHECK: { i8, i8 } @enum_id_2(i1 noundef zeroext %x.0, i8 %x.1)
|
|
#[no_mangle]
|
|
pub fn enum_id_2(x: Option<u8>) -> Option<u8> {
|
|
x
|
|
}
|