Improve codegen for `align_offset`
In this PR the `align_offset` implementation is changed/improved to produce better code in certain scenarios such as when pointer type is has a stride of 1 or when building for low optimisation levels.
While these changes do not achieve the "ideal" codegen referenced in #75579, it gets significantly closer to it. I’m not actually sure if the codegen can actually be much better with this function returning the offset, rather than the aligned pointer.
See the descriptions for separate commits for further information.
Add `as_uninit`-like methods to pointer types and unify documentation of `as_ref` methods
This adds a convenient method to retrieve a `&(mut) [MaybeUninit<T>]` from slice pointers (`*const [T]`, `*mut [T]`, `NonNull<[T]>`). See also https://github.com/rust-lang/wg-allocators/issues/66#issuecomment-671789105.
~I'll add a tracking issue as soon as it's reviewed and CI passed.~
Tracking Issue: #75402
r? @RalfJung
Fix example in `NonNull::as_uninit_slice`
Rename feature gate to "ptr_as_uninit"
Make methods more consistent with already stable methods
Make `pointer::as_uninit_slice` return an `Option`
Fix placement for `// SAFETY` section
Add `as_uninit_ref` and `as_uninit_mut` to pointers
Fix doctest
Update tracking issue
Fix doc links
Apply suggestions from review
Make wording about counterparts consistent
Fix doc links
Improve documentation
Fix doc-tests
Fix doc links ... again
Apply suggestions from review
Apply suggestions from Review
Apply suggestion from review to all affected files
Add missing words in safety sections in `as_uninit_slice_mut`
Fix safety-comment in `NonNull::as_uninit_slice_mut`
Previously checking for `pmoda == 0` would get LLVM to generate branchy
code, when, for `stride = 1` the offset can be computed without such a
branch by doing effectively a `-p % a`.
For well-known (constant) alignments, with the new ordering of these
conditionals, we end up generating 2 to 3 cheap instructions on x86_64:
movq %rdi, %rax
negl %eax
andl $7, %eax
instead of 5+ as previously.
For unknown alignments the new code also generates just 3 instructions:
negq %rdi
leaq -1(%rsi), %rax
andq %rdi, %rax
At opt-level <= 1, the methods such as `wrapping_mul` are not being
inlined, causing significant bloating and slowdowns of the
implementation at these optimisation levels.
With use of these intrinsics, the codegen of this function at
-Copt_level=1 is the same as it is at -Copt_level=3.
Add safety section to `NonNull::as_*` method docs
This basically adds the safety section of `*mut T::as_{ref,mut}` to the
same methods on `NonNull` with minor modifications to fit the
differences.
Part of #48929.
This basically adds the safety section of `*mut T::as_{ref,mut}` to the
same methods on `NonNull` with minor modifications to fit the
differences.
Part of #48929.
