Commit graph

895 commits

Author SHA1 Message Date
Matthias Krüger
b8c55b438d
Rollup merge of #140660 - RalfJung:more-order, r=WaffleLapkin
remove 'unordered' atomic intrinsics

As their doc comment already indicates, these operations do not currently have a place in our memory model. The intrinsics were introduced to support a hack in compiler-builtins, but that hack recently got removed (see https://github.com/rust-lang/compiler-builtins/issues/788).
2025-05-10 16:26:02 +02:00
Ralf Jung
79dfd0a472 remove 'unordered' atomic intrinsics 2025-05-09 17:39:52 +02:00
Michael Goulet
3a1ee645ca Resolve instance for SymFn in global/naked asm 2025-05-05 13:17:35 +00:00
Matthias Krüger
555df301f8
Rollup merge of #134232 - bjorn3:naked_asm_improvements, r=wesleywiser
Share the naked asm impl between cg_ssa and cg_clif

This was introduced in https://github.com/rust-lang/rust/pull/128004.
2025-04-30 17:27:57 +02:00
Andrew Zhogin
c366756a85 AsyncDrop implementation using shim codegen of async_drop_in_place::{closure}, scoped async drop added. 2025-04-28 16:23:13 +07:00
bjorn3
421f22e8bf Pass &mut self to codegen_global_asm 2025-04-14 09:38:04 +00:00
bjorn3
94e95f389c Make codegen_naked_asm public
This allows it to be reused by codegen backends that don't use cg_ssa
like cg_clif.
2025-04-14 09:38:04 +00:00
bjorn3
e2e96fa14e Pass MonoItemData to MonoItem::define 2025-04-14 09:38:03 +00:00
bjorn3
a73eba99f7 Move codegen_naked_asm call up into MonoItem::define 2025-04-14 09:38:02 +00:00
bjorn3
764d3a50a3 Make codegen_naked_asm retrieve the MIR Body itself 2025-04-14 09:36:20 +00:00
bjorn3
1988de5c37 Only require a CodegenCx for codegen_naked_asm 2025-04-14 09:34:23 +00:00
bjorn3
ffdc292cca Don't begin defining a function when codegening a naked function
While LLVM is rather permissive in this regards, some other codegen
backends demand that once you declare a function for definition you
actually define contents of the function, which doesn't happen for naked
functions as we actually generate assembly for them.
2025-04-14 09:34:21 +00:00
bjorn3
03f4e886dc Handle protected visibility in codegen_naked_asm 2025-04-14 09:25:31 +00:00
bjorn3
14491b0f43 Use START_BLOCK in codegen_naked_asm 2025-04-14 09:25:31 +00:00
bors
7d7de5bf3c Auto merge of #139088 - spastorino:ergonomic-ref-counting-2, r=nikomatsakis
Ergonomic ref counting: optimize away clones when possible

This PR build on top of https://github.com/rust-lang/rust/pull/134797. It optimizes codegen of ergonomic ref-counting when the type being `use`d is only known to be copy after monomorphization. We avoid codening a clone and generate bitwise copy instead.

RFC: https://github.com/rust-lang/rfcs/pull/3680
Tracking issue: https://github.com/rust-lang/rust/issues/132290
Project goal: https://github.com/rust-lang/rust-project-goals/issues/107

r? `@nikomatsakis`

This PR could better sit on top of https://github.com/rust-lang/rust/pull/131650 but as it did not land yet I've decided to just do minimal changes. It may be the case that doing what I'm doing regress the performance and we may need to go the full route of https://github.com/rust-lang/rust/pull/131650.
cc `@saethlin` in this regard.
2025-04-10 09:08:23 +00:00
Scott McMurray
502f7f9c24 Address PR feedback 2025-04-07 18:12:06 -07:00
Santiago Pastorino
4a0ea02e3a
Only clone mir body if tcx.features().ergonomic_clones() 2025-04-07 16:53:12 -03:00
Santiago Pastorino
9f69c66a47
Optimize codegen of use values that are copy post monomorphization 2025-04-07 16:53:11 -03:00
Santiago Pastorino
b9159e09a7
Use a local var for tcx 2025-04-07 16:53:11 -03:00
Scott McMurray
1f06a6a252 Tell LLVM about impossible niche tags 2025-04-05 19:54:47 -07:00
bors
068609ce76 Auto merge of #138601 - RalfJung:wasm-abi-fcw, r=alexcrichton
add FCW to warn about wasm ABI transition

See https://github.com/rust-lang/rust/issues/122532 for context: the "C" ABI on wasm32-unk-unk will change. The goal of this lint is to warn about any function definition and calls whose behavior will be affected by the change. My understanding is the following:
- scalar arguments are fine
  - including 128 bit types, they get passed as two `i64` arguments in both ABIs
- `repr(C)` structs (recursively) wrapping a single scalar argument are fine (unless they have extra padding due to over-alignment attributes)
- all return values are fine

`@bjorn3` `@alexcrichton` `@Manishearth` is that correct?

I am making this a "show up in future compat reports" lint to maximize the chances people become aware of this. OTOH this likely means warnings for most users of Diplomat so maybe we shouldn't do this?

IIUC, wasm-bindgen should be unaffected by this lint as they only pass scalar types as arguments.

Tracking issue: https://github.com/rust-lang/rust/issues/138762
Transition plan blog post: https://github.com/rust-lang/blog.rust-lang.org/pull/1531

try-job: dist-various-2
2025-03-26 00:06:46 +00:00
Ralf Jung
072ccce553 make -Zwasm-c-abi=legacy suppress the lint 2025-03-25 08:22:35 +01:00
bors
e61403aa4c Auto merge of #138634 - saethlin:repeated-uninit, r=scottmcm,oli-obk
Lower to a memset(undef) when Rvalue::Repeat repeats uninit

Fixes https://github.com/rust-lang/rust/issues/138625.

It is technically correct to just do nothing. But if we actually do nothing, we may miss that this is de-initializing something, so instead we just lower to a single memset that writes undef. This is still superior to the memcpy loop, in both quality of code we hand to the backend and LLVM's final output.
2025-03-25 02:09:15 +00:00
bors
1df5affaca Auto merge of #133984 - DaniPopes:scmp-ucmp, r=scottmcm
Lower BinOp::Cmp to llvm.{s,u}cmp.* intrinsics

Lowers `mir::BinOp::Cmp` (`three_way_compare` intrinsic) to the corresponding LLVM `llvm.{s,u}cmp.i8.*` intrinsics.

These are the intrinsics mentioned in https://github.com/rust-lang/rust/pull/118310, which are now available in LLVM 19.

I couldn't find any follow-up PRs/discussions about this, please let me know if I missed something.

r? `@scottmcm`
2025-03-24 22:53:12 +00:00
Ben Kimock
8e7d8ddffe Lower to a memset(undef) when Rvalue::Repeat repeats uninit 2025-03-19 23:57:49 -04:00
Yotam Ofek
36f6bc5e3d Flatten ifs in rustc_codegen_ssa 2025-03-17 18:56:52 +00:00
许杰友 Jieyou Xu (Joe)
e714c3be9f
Rollup merge of #138549 - scottmcm:option-ssa, r=saethlin
Fix the OperandRef type for NullOp::{UbChecks,ContractChecks}

Stumbled on this while looking at something totally unrelated 🙃

r? saethlin
2025-03-16 13:19:53 +08:00
Scott McMurray
3d42541313 Fix the OperandRef type for NullOp::{UbChecks,ContractChecks} 2025-03-15 19:39:15 -07:00
Michael Goulet
b88f85a410 Stop relying on rustc_type_ir in non-type-system crates 2025-03-15 06:42:48 +00:00
Jacob Pratt
91e4bab25f
Rollup merge of #138398 - RalfJung:atomic-intrinsics-provenance, r=nnethercote
atomic intrinsics: clarify which types are supported and (if applicable) what happens with provenance

The provenance semantics match what Miri implements and what the `AtomicPtr` API expects.
2025-03-14 01:37:32 -04:00
bors
addae0705c Auto merge of #138391 - scottmcm:SSA-discriminants, r=WaffleLapkin
Don't `alloca` just to look at a discriminant

Today we're making LLVM do a bunch of extra work when you match on trivial stuff like `Option<bool>` or `ControlFlow<u8>`.

This PR changes that so that simple types like `Option<u32>` or `Result<(), Box<Error>>` can stay as `OperandValue::ScalarPair` and we can still read the discriminant from them, rather than needing to write them into memory to have a `PlaceValue` just to get the discriminant out.

Fixes #137503
2025-03-14 00:42:31 +00:00
bors
93257e2d20 Auto merge of #138450 - matthiaskrgr:rollup-4im25vf, r=matthiaskrgr
Rollup of 6 pull requests

Successful merges:

 - #137816 (attempt to support `BinaryFormat::Xcoff` in `naked_asm!`)
 - #138109 (make precise capturing args in rustdoc Json typed)
 - #138343 (Enable `f16` tests for `powf`)
 - #138356 (bump libc to 0.2.171 to fix xous)
 - #138371 (Update compiletest's `has_asm_support` to match rustc)
 - #138404 (Cleanup sysroot locating a bit)

r? `@ghost`
`@rustbot` modify labels: rollup
2025-03-13 13:34:28 +00:00
Matthias Krüger
762acf53cb
Rollup merge of #137816 - folkertdev:naked-asm-xcoff, r=Noratrieb
attempt to support `BinaryFormat::Xcoff` in `naked_asm!`

Fixes https://github.com/rust-lang/rust/issues/137219

So, the inline assembly support for xcoff is extremely limited. The LLVM [XCOFFAsmParser](1b25c0c4da/llvm/lib/MC/MCParser/XCOFFAsmParser.cpp) does not support many of the attributes that LLVM itself emits, and that should exist based on [the assembler docs](https://www.ibm.com/docs/en/ssw_aix_71/assembler/assembler_pdf.pdf). It also does accept some that should not exist based on those docs.

So, I've tried to do the best I can given those limitations. At least it's better than emitting the directives for elf and having that fail somewhere deep in LLVM. Given that inline assembly for this target is incomplete (under `asm_experimental_arch`), I think that's OK (and again I don't see how we can do better given the limitations in LLVM).

r? ```@Noratrieb``` (given that you reviewed https://github.com/rust-lang/rust/pull/136637)

It seems reasonable to ping the [`powerpc64-ibm-aix` target maintainers](https://doc.rust-lang.org/rustc/platform-support/aix.html), hopefully they have thoughts too: ```@daltenty``` ```@gilamn5tr```
2025-03-13 11:28:20 +01:00
Scott McMurray
2b15dd1ddd Add more comments to discriminant calculations. 2025-03-13 00:39:18 -07:00
Ralf Jung
88b206d582 atomic intrinsics: clarify which types are supported and (if applicable) what happens with provenance 2025-03-13 08:14:34 +01:00
Scott McMurray
143f39362a Don't alloca just to look at a discriminant
Today we're making LLVM do a bunch of extra work for every enum you match on, even trivial stuff like `Option<bool>`.  Let's not.
2025-03-12 00:56:43 -07:00
Folkert de Vries
c0957ef45a
naked functions: on windows emit .endef without the symbol name
also add test with `fastcall`, which on i686 uses a different mangling scheme
2025-03-11 00:27:32 +01:00
bors
446649d463 Auto merge of #137513 - scottmcm:identity-transmute, r=saethlin
Don't re-`assume` in `transmute`s that don't change niches

I noticed in nightly 2025-02-21 that `transmute` is emitting way more `assume`s than necessary for newtypes.

For example, the three transmutes in <https://rust.godbolt.org/z/fW1KaTc4o> emits
```rust
define noundef range(i32 1, 0) i32 `@repeatedly_transparent_transmute(i32` noundef range(i32 1, 0) %_1) unnamed_addr {
start:
  %0 = sub i32 %_1, 1
  %1 = icmp ule i32 %0, -2
  call void `@llvm.assume(i1` %1)
  %2 = sub i32 %_1, 1
  %3 = icmp ule i32 %2, -2
  call void `@llvm.assume(i1` %3)
  %4 = sub i32 %_1, 1
  %5 = icmp ule i32 %4, -2
  call void `@llvm.assume(i1` %5)
  %6 = sub i32 %_1, 1
  %7 = icmp ule i32 %6, -2
  call void `@llvm.assume(i1` %7)
  %8 = sub i32 %_1, 1
  %9 = icmp ule i32 %8, -2
  call void `@llvm.assume(i1` %9)
  %10 = sub i32 %_1, 1
  %11 = icmp ule i32 %10, -2
  call void `@llvm.assume(i1` %11)
  ret i32 %_1
}
```

But those are all just newtypes that don't change size or niches, so none of it's needed.

After this PR it's down to just
```rust
define noundef range(i32 1, 0) i32 `@repeatedly_transparent_transmute(i32` noundef range(i32 1, 0) %_1) unnamed_addr {
start:
  ret i32 %_1
}
```
because none of those `assume`s in the original actually did anything.

(Transmuting to something with a difference niche, though, still has the assumes -- the other tests continue to pass checking that.)
2025-03-09 01:25:48 +00:00
bors
efea9896f5 Auto merge of #137500 - scottmcm:trunc-br, r=saethlin
Use `trunc nuw`+`br` for 0/1 branches even in optimized builds

Rather than needing to use `switch` for them to include the `unreachable` arm.
2025-03-08 19:01:10 +00:00
Folkert de Vries
f35bda3997
support XCOFF in naked_asm! 2025-03-08 14:10:29 +01:00
Matthias Krüger
63c548d82c
Rollup merge of #137549 - oli-obk:llvm-ffi, r=davidtwco
Clean up various LLVM FFI things in codegen_llvm

cc ```@ZuseZ4``` I touched some autodiff parts

The major change of this PR is [bfd88ce](https://github.com/rust-lang/rust/pull/137549/commits/bfd88cead0dd79717f123ad7e9a26ecad88653cb) which makes `CodegenCx` generic just like `GenericBuilder`

The other commits mostly took advantage of the new feature of making extern functions safe, but also just used some wrappers that were already there and shrunk unsafe blocks.

best reviewed commit-by-commit
2025-03-07 19:15:34 +01:00
Scott McMurray
d9432acfe1 Use trunc nuw+br for 0/1 branches even in optimized builds
Rather than needing to use `switch` for them to include the `unreachable` arm
2025-03-06 22:25:49 -08:00
DaniPopes
58c10c66c1
Lower BinOp::Cmp to llvm.{s,u}cmp.* intrinsics
Lowers `mir::BinOp::Cmp` (`three_way_compare` intrinsic) to the corresponding
LLVM `llvm.{s,u}cmp.i8.*` intrinsics, added in LLVM 19.
2025-03-06 22:29:05 +08:00
Matthias Krüger
70b9968d1e
Rollup merge of #137894 - compiler-errors:no-scalar-pair-opt, r=oli-obk
Revert "store ScalarPair via memset when one side is undef and the other side can be memset"

cc #137892
reverts #135335

r? oli-obk
2025-03-03 20:47:12 +01:00
Michael Goulet
a59a8f9e75 Revert "Auto merge of #135335 - oli-obk:push-zxwssomxxtnq, r=saethlin"
This reverts commit a7a6c64a65, reversing
changes made to ebbe63891f.
2025-03-02 18:52:48 +00:00
Ralf Jung
aac65f562b rename BackendRepr::Vector → SimdVector 2025-02-28 17:17:45 +01:00
Oli Scherer
d4379d2afd Remove an unnecessary lifetime 2025-02-24 15:05:56 +00:00
David Wood
5afa6a111b
ssa/mono: deduplicate type_has_metadata
The implementation of the `type_has_metadata` function is duplicated in
`rustc_codegen_ssa` and `rustc_monomorphize`, so move this to
`rustc_middle`.
2025-02-24 08:08:23 +00:00
Scott McMurray
23c6b93de8 Don't re-assume in transmutes that don't change niches 2025-02-23 23:18:04 -08:00
Jacob Pratt
42014b44b3
Rollup merge of #137505 - tgross35:builtins-cannot-call-error, r=compiler-errors
Add a span to `CompilerBuiltinsCannotCall`

Currently, this error emit a diagnostic with no context like:

    error: `compiler_builtins` cannot call functions through upstream monomorphizations; encountered invalid call from `<math::libm::support::hex_float::Hexf<i32> as core::fmt::LowerHex>::fmt` to `core::fmt::num::<impl core::fmt::LowerHex for i32>::fmt`

With this change, it at least usually points to the problematic function:

    error: `compiler_builtins` cannot call functions through upstream monomorphizations; encountered invalid call from `<math::libm::support::hex_float::Hexf<i32> as core::fmt::LowerHex>::fmt` to `core::fmt::num::<impl core::fmt::LowerHex for i32>::fmt`
       --> src/../libm/src/math/support/hex_float.rs:270:5
        |
    270 |     fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        |
2025-02-24 02:11:38 -05:00