Introduce support for `async gen` blocks
I'm delighted to demonstrate that `async gen` block are not very difficult to support. They're simply coroutines that yield `Poll<Option<T>>` and return `()`.
**This PR is WIP and in draft mode for now** -- I'm mostly putting it up to show folks that it's possible. This PR needs a lang-team experiment associated with it or possible an RFC, since I don't think it falls under the jurisdiction of the `gen` RFC that was recently authored by oli (https://github.com/rust-lang/rfcs/pull/3513, https://github.com/rust-lang/rust/issues/117078).
### Technical note on the pre-generator-transform yield type:
The reason that the underlying coroutines yield `Poll<Option<T>>` and not `Poll<T>` (which would make more sense, IMO, for the pre-transformed coroutine), is because the `TransformVisitor` that is used to turn coroutines into built-in state machine functions would have to destructure and reconstruct the latter into the former, which requires at least inserting a new basic block (for a `switchInt` terminator, to match on the `Poll` discriminant).
This does mean that the desugaring (at the `rustc_ast_lowering` level) of `async gen` blocks is a bit more involved. However, since we already need to intercept both `.await` and `yield` operators, I don't consider it much of a technical burden.
r? `@ghost`
never_patterns: Parse match arms with no body
Never patterns are meant to signal unreachable cases, and thus don't take bodies:
```rust
let ptr: *const Option<!> = ...;
match *ptr {
None => { foo(); }
Some(!),
}
```
This PR makes rustc accept the above, and enforces that an arm has a body xor is a never pattern. This affects parsing of match arms even with the feature off, so this is delicate. (Plus this is my first non-trivial change to the parser).
~~The last commit is optional; it introduces a bit of churn to allow the new suggestions to be machine-applicable. There may be a better solution? I'm not sure.~~ EDIT: I removed that commit
r? `@compiler-errors`
Avoid adding builtin functions to `symbols.o`
We found performance regressions in #113923. The problem seems to be that `--gc-sections` does not remove these symbols. I tested that lld removes these symbols, but ld and gold do not.
I found that `used` adds symbols to `symbols.o` at 3e202ead60/compiler/rustc_codegen_ssa/src/back/linker.rs (L1786-L1791).
The PR removes builtin functions.
Note that under LTO, ld still preserves these symbols. (lld will still remove them.)
The first commit also fixes#118559. But I think the second commit also makes sense.
compile-time evaluation: detect writes through immutable pointers
This has two motivations:
- it unblocks https://github.com/rust-lang/rust/pull/116745 (and therefore takes a big step towards `const_mut_refs` stabilization), because we can now detect if the memory that we find in `const` can be interned as "immutable"
- it would detect the UB that was uncovered in https://github.com/rust-lang/rust/pull/117905, which was caused by accidental stabilization of `copy` functions in `const` that can only be called with UB
When UB is detected, we emit a future-compat warn-by-default lint. This is not a breaking change, so completely in line with [the const-UB RFC](https://rust-lang.github.io/rfcs/3016-const-ub.html), meaning we don't need t-lang FCP here. I made the lint immediately show up for dependencies since it is nearly impossible to even trigger this lint without `const_mut_refs` -- the accidentally stabilized `copy` functions are the only way this can happen, so the crates that popped up in #117905 are the only causes of such UB (in the code that crater covers), and the three cases of UB that we know about have all been fixed in their respective crates already.
The way this is implemented is by making use of the fact that our interpreter is already generic over the notion of provenance. For CTFE we now use the new `CtfeProvenance` type which is conceptually an `AllocId` plus a boolean `immutable` flag (but packed for a more efficient representation). This means we can mark a pointer as immutable when it is created as a shared reference. The flag will be propagated to all pointers derived from this one. We can then check the immutable flag on each write to reject writes through immutable pointers.
I just hope perf works out.
`riscv32` platform support
This PR adds the following RISCV targets to the tier 2 list of targets:
- riscv32imafc-unknown-none-elf
- riscv32im-unknown-none-elf
The rationale behind adding them directly to tier 2, is that the other bare metal targets already exist at tier 2, and these new targets are the same with an additional target feature enabled.
As well as the additional targets, this PR fills out the platform support document(s) that were previously missing.
~~The RISC-V bare metal targets don't currently have a platform support document, but this will change soon as the RISC-V team from the Rust-embedded working group will maintain these once https://github.com/davidtwco/rust/pull/1 is merged (and `@davidtwco's` upstream PR is merged after). For the time being you can cc myself or any other member of the RISC-V team: https://github.com/orgs/rust-embedded/teams/riscv.~~
> A tier 2 target must have value to people other than its maintainers. (It may still be a niche target, but it must not be exclusively useful for an inherently closed group.)
RISC-V is an open specification, used and accessible to anyone including individuals.
> A tier 2 target must have a designated team of developers (the "target maintainers") available to consult on target-specific build-breaking issues, or if necessary to develop target-specific language or library implementation details. This team must have at least 2 developers.
This rust-embedded working group's [RISCV team](https://github.com/orgs/rust-embedded/teams/riscv) will maintain these targets.
> The target must not place undue burden on Rust developers not specifically concerned with that target. Rust developers are expected to not gratuitously break a tier 2 target, but are not expected to become experts in every tier 2 target, and are not expected to provide target-specific implementations for every tier 2 target.
I don't forsee this being an issue, the RISCV team will ensure we avoid undue burden for the general Rust community.
> The target must provide documentation for the Rust community explaining how to build for the target using cross-compilation, and explaining how to run tests for the target. If at all possible, this documentation should show how to run Rust programs and tests for the target using emulation, to allow anyone to do so. If the target cannot be feasibly emulated, the documentation should explain how to obtain and work with physical hardware, cloud systems, or equivalent.
There are links to resources we maintain in the re wg org in the platform support document.
> The target must document its baseline expectations for the features or versions of CPUs, operating systems, libraries, runtime environments, and similar.
Documented in the platform support document.
> If introducing a new tier 2 or higher target that is identical to an existing Rust target except for the baseline expectations for the features or versions of CPUs, operating systems, libraries, runtime environments, and similar, then the proposed target must document to the satisfaction of the approving teams why the specific difference in baseline expectations provides sufficient value to justify a separate target.
New target features in RISCV can drastically change the capability of a CPU, hence the need for a separate target to support different variants. We aim to support any ratified RISCV extensions.
> Tier 2 targets must not leave any significant portions of core or the standard library unimplemented or stubbed out, unless they cannot possibly be supported on the target.
`core` is fully implemented.
> The code generation backend for the target should not have deficiencies that invalidate Rust safety properties, as evaluated by the Rust compiler team. (This requirement does not apply to arbitrary security enhancements or mitigations provided by code generation backends, only to those properties needed to ensure safe Rust code cannot cause undefined behavior or other unsoundness.) If this requirement does not hold, the target must clearly and prominently document any such limitations as part of the target's entry in the target tier list, and ideally also via a failing test in the testsuite. The Rust compiler team must be satisfied with the balance between these limitations and the difficulty of implementing the necessary features.
RISCV is a well-established and well-maintained LLVM backend. To the best of my knowledge, the backend won't cause the generated code to have undefined behaviour.
> If the target supports C code, and the target has an interoperable calling convention for C code, the Rust target must support that C calling convention for the platform via extern "C". The C calling convention does not need to be the default Rust calling convention for the target, however.
The C calling convention is supported by RISCV.
> The target must build reliably in CI, for all components that Rust's CI considers mandatory.
For the last 4-5 years many of these RISCV targets have been building in CI without any known issues.
> The approving teams may additionally require that a subset of tests pass in CI, such as enough to build a functional "hello world" program, ./x.py test --no-run, or equivalent "smoke tests". In particular, this requirement may apply if the target builds host tools, or if the tests in question provide substantial value via early detection of critical problems.
Not applicable, in the future we may wish to add qemu tests but this is out of scope for now.
> Building the target in CI must not take substantially longer than the current slowest target in CI, and should not substantially raise the maintenance burden of the CI infrastructure. This requirement is subjective, to be evaluated by the infrastructure team, and will take the community importance of the target into account.
To the best of my knowledge, this will not induce a burden on the current CI infra.
> Tier 2 targets should, if at all possible, support cross-compiling. Tier 2 targets should not require using the target as the host for builds, even if the target supports host tools.
Cross-compilation is supported and documented in the platform support document.
> In addition to the legal requirements for all targets (specified in the tier 3 requirements), because a tier 2 target typically involves the Rust project building and supplying various compiled binaries, incorporating the target and redistributing any resulting compiled binaries (e.g. built libraries, host tools if any) must not impose any onerous license requirements on any members of the Rust project, including infrastructure team members and those operating CI systems. This is a subjective requirement, to be evaluated by the approving teams.
There are no additional license issues to worry about.
> Tier 2 targets must not impose burden on the authors of pull requests, or other developers in the community, to ensure that tests pass for the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on tests failing for the target. Do not send automated messages or notifications (via any medium, including via `@)` to a PR author or others involved with a PR regarding the PR breaking tests on a tier 2 target, unless they have opted into such messages.
The RISCV team agrees not to do this.
> The target maintainers should regularly run the testsuite for the target, and should fix any test failures in a reasonably timely fashion.
The RISCV team will fix any issues in a timely manner.
Use `unwinding` crate for unwinding on Xous platform
This patch adds support for using [unwinding](https://github.com/nbdd0121/unwinding) on platforms where libunwinding isn't viable. An example of such a platform is `riscv32imac-unknown-xous-elf`.
### Background
The Rust project maintains a fork of llvm at [llvm-project](https://github.com/rust-lang/llvm-project/) where it applies patches on top of the llvm project. This mostly seems to be to get unwinding support for the SGX project, and there may be other patches that I'm unaware of.
There is a lot of machinery in the build system to support compiling `libunwind` on other platforms, and I needed to add additional patches to llvm in order to add support for Xous.
Rather than continuing down this path, it seemed much easier to use a Rust-based library. The `unwinding` crate by `@nbdd0121` fits this description perfectly.
### Future work
This could potentially replace the custom patches for `libunwind` on other platforms such as SGX, and could enable unwinding support on many more exotic platforms.
### Anti-goals
This is not designed to replace `libunwind` on tier-one platforms or those where unwinding support already exists. There is already a well-established approach for unwinding there. Instead, this aims to enable unwinding on new platforms where C++ code may be difficult to compile.
Add `unwinding` as a permitted dependency of rustc, as it is now used as
part of panic unwinding within platforms such as Xous.
Signed-off-by: Sean Cross <sean@xobs.io>
Add support for making lib features internal
We have the notion of an "internal" lang feature: a feature that is never intended to be stabilized, and using which can cause ICEs and other issues without that being considered a bug.
This extends that idea to lib features as well. It is an alternative to https://github.com/rust-lang/rust/pull/115623: instead of using an attribute to declare lib features internal, we simply do this based on the name. Everything ending in `_internals` or `_internal` is considered internal.
Then we rename `core_intrinsics` to `core_intrinsics_internal`, which fixes https://github.com/rust-lang/rust/issues/115597.
Add support for `gen fn`
This builds on #116447 to add support for `gen fn` functions. For the most part we follow the same approach as desugaring `async fn`, but replacing `Future` with `Iterator` and `async {}` with `gen {}` for the body.
The version implemented here uses the return type of a `gen fn` as the yield type. For example:
```rust
gen fn count_to_three() -> i32 {
yield 1;
yield 2;
yield 3;
}
```
In the future, I think we should experiment with a syntax like `gen fn count_to_three() yield i32 { ... }`, but that can go in another PR.
cc `@oli-obk` `@compiler-errors`
Rollup of 5 pull requests
Successful merges:
- #118495 (Restrict what symbols can be used in `#[diagnostic::on_unimplemented]` format strings)
- #118540 (codegen, miri: fix computing the offset of an unsized field in a packed struct)
- #118551 (more targeted errors when extern types end up in places they should not)
- #118573 (rustc: Harmonize `DefKind` and `DefPathData`)
- #118586 (Improve example in `slice::windows()` doc)
r? `@ghost`
`@rustbot` modify labels: rollup
miri: support 'promising' alignment for symbolic alignment check
Then use that ability in `slice::align_to`, so that even with `-Zmiri-symbolic-alignment-check`, it no longer has to return spuriously empty "middle" parts.
Fixes https://github.com/rust-lang/miri/issues/3068
Report errors in jobserver inherited through environment variables
This pr attempts to catch situations, when jobserver exists, but is not being inherited.
r? `@petrochenkov`
