Fix RustAnalyzer discovery of rustc's `stable_mir` crate
This fixes issues with RustAnalyzer not finding `stable_mir` crate since RA discovery traverses the dependency graph of `rustc_driver` crate.
This change also aligns with the long term architecture plan for these crates, since we are moving towards having stable_mir depend on rustc_smir and not the other way around. See [this doc](https://hackmd.io/jBRkZLqAQL2EVgwIIeNMHg) for more details.
I believe a similar function will come handy eventually for `stable_mir` users, but I'm keeping it as part of `rustc_internal` since its current format initializes the StableMir context and requires `TyCtxt`.
Finally, I added the `rustc_internal` module re-export under a feature since the APIs from this module shall not be stabilized.
Removing rustc_type_ir in the rustc_infer codebase
cc #138449
This is a second refactoring of rustc_type_ir to use rustc_middle instead, this time that's for rustc_infer
Initial support for dynamically linked crates
This PR is an initial implementation of [rust-lang/rfcs#3435](https://github.com/rust-lang/rfcs/pull/3435) proposal.
### component 1: interface generator
Interface generator - a tool for generating a stripped version of crate source code. The interface is like a C header, where all function bodies are omitted. For example, initial crate:
```rust
#[export]
#[repr(C)]
pub struct S {
pub x: i32
}
#[export]
pub extern "C" fn foo(x: S) {
m1::bar(x);
}
pub fn bar(x: crate::S) {
// some computations
}
```
generated interface:
```rust
#[export]
#[repr(C)]
pub struct S {
pub x: i32,
}
#[export]
pub extern "C" fn foo(x: S);
pub fn bar(x: crate::S);
```
The interface generator was implemented as part of the pretty-printer. Ideally interface should only contain exportable items, but here is the first problem:
- pass for determining exportable items relies on privacy information, which is totally available only in HIR
- HIR pretty-printer uses pseudo-code(at least for attributes)
So, the interface generator was implemented in AST. This has led to the fact that non-exportable items cannot be filtered out, but I don't think this is a major issue at the moment.
To emit an interface use a new `sdylib` crate type which is basically the same as `dylib`, but it doesn't contain metadata, and also produces the interface as a second artifact. The current interface name is `lib{crate_name}.rs`.
#### Why was it decided to use a design with an auto-generated interface?
One of the main objectives of this proposal is to allow building the library and the application with different compiler versions. This requires either a metadata format compatible across rustc versions or some form of a source code. The option with a stable metadata format has not been investigated in detail, but it is not part of RFC either. Here is the the related discussion: https://github.com/rust-lang/rfcs/pull/3435#discussion_r1202872373
Original proposal suggests using the source code for the dynamic library and all its dependencies. Metadata is obtained from `cargo check`. I decided to use interface files since it is more or less compatible with the original proposal, but also allows users to hide the source code.
##### Regarding the design with interfaces
in Rust, files generally do not have a special meaning, unlike C++. A translation unit i.e. a crate is not a single file, it consists of modules. Modules, in turn, can be declared either in one file or divided into several. That's why the "interface file" isn't a very coherent concept in Rust. I would like to avoid adding an additional level of complexity for users until it is proven necessary. Therefore, the initial plan was to make the interfaces completely invisible to users i. e. make them auto-generated. I also planned to put them in the dylib, but this has not been done yet. (since the PR is already big enough, I decided to postpone it)
There is one concern, though, which has not yet been investigated(https://github.com/rust-lang/rust/pull/134767#issuecomment-2736471828):
> Compiling the interface as pretty-printed source code doesn't use correct macro hygiene (mostly relevant to macros 2.0, stable macros do not affect item hygiene). I don't have much hope for encoding hygiene data in any stable way, we should rather support a way for the interface file to be provided manually, instead of being auto-generated, if there are any non-trivial requirements.
### component 2: crate loader
When building dynamic dependencies, the crate loader searches for the interface in the file system, builds the interface without codegen and loads it's metadata. Routing rules for interface files are almost the same as for `rlibs` and `dylibs`. Firstly, the compiler checks `extern` options and then tries to deduce the path himself.
Here are the code and commands that corresponds to the compilation process:
```rust
// simple-lib.rs
#![crate_type = "sdylib"]
#[extern]
pub extern "C" fn foo() -> i32 {
42
}
```
```rust
// app.rs
extern crate simple_lib;
fn main() {
assert!(simple_lib::foo(), 42);
}
```
```
// Generate interface, build library.
rustc +toolchain1 lib.rs
// Build app. Perhaps with a different compiler version.
rustc +toolchain2 app.rs -L.
```
P.S. The interface name/format and rules for file system routing can be changed further.
### component 3: exportable items collector
Query for collecting exportable items. Which items are exportable is defined [here](https://github.com/m-ou-se/rfcs/blob/export/text/0000-export.md#the-export-attribute) .
### component 4: "stable" mangling scheme
The mangling scheme proposed in the RFC consists of two parts: a mangled item path and a hash of the signature.
#### mangled item path
For the first part of the symbol it has been decided to reuse the `v0` mangling scheme as it much less dependent on compiler internals compared to the `legacy` scheme.
The exception is disambiguators (https://doc.rust-lang.org/rustc/symbol-mangling/v0.html#disambiguator):
For example, during symbol mangling rustc uses a special index to distinguish between two impls of the same type in the same module(See `DisambiguatedDefPathData`). The calculation of this index may depend on private items, but private items should not affect the ABI. Example:
```rust
#[export]
#[repr(C)]
pub struct S<T>(pub T);
struct S1;
pub struct S2;
impl S<S1> {
extern "C" fn foo() -> i32 {
1
}
}
#[export]
impl S<S2> {
// Different symbol names can be generated for this item
// when compiling the interface and source code.
pub extern "C" fn foo() -> i32 {
2
}
}
```
In order to make disambiguation independent of the compiler version we can assign an id to each impl according to their relative order in the source code.
The second example is `StableCrateId` which is used to disambiguate different crates. `StableCrateId` consists of crate name, `-Cmetadata` arguments and compiler version. At the moment, I have decided to keep only the crate name, but a more consistent approach to crate disambiguation could be added in the future.
Actually, there are more cases where such disambiguation can be used. For instance, when mangling internal rustc symbols, but it also hasn't been investigated in detail yet.
#### hash of the signature
Exportable functions from stable dylibs can be called from safe code. In order to provide type safety, 128 bit hash with relevant type information is appended to the symbol ([description from RFC](https://github.com/m-ou-se/rfcs/blob/export/text/0000-export.md#name-mangling-and-safety)). For now, it includes:
- hash of the type name for primitive types
- for ADT types with public fields the implementation follows [this](https://github.com/m-ou-se/rfcs/blob/export/text/0000-export.md#types-with-public-fields) rules
`#[export(unsafe_stable_abi = "hash")]` syntax for ADT types with private fields is not yet implemented.
Type safety is a subtle thing here. I used the approach from RFC, but there is the ongoing research project about it. [https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html](https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html)
### Unresolved questions
Interfaces:
1. Move the interface generator to HIR and add an exportable items filter.
2. Compatibility of auto-generated interfaces and macro hygiene.
3. There is an open issue with interface files compilation: https://github.com/rust-lang/rust/pull/134767#issuecomment-2736471828
4. Put an interface into a dylib.
Mangling scheme:
1. Which information is required to ensure type safety and how should it be encoded? ([https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html](https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html))
2. Determine all other possible cases, where path disambiguation is used. Make it compiler independent.
We also need a semi-stable API to represent types. For example, the order of fields in the `VariantDef` must be stable. Or a semi-stable representation for AST, which ensures that the order of the items in the code is preserved.
There are some others, mentioned in the proposal.
Refactor rustc_on_unimplemented's filter parser
Followup to https://github.com/rust-lang/rust/pull/139091; I plan on moving most of this code into `rustc_attr_parsing` at some point, but want to land this separately first.
I have taken care to preserve the original behavior as much as I could:
- All but one of the new error variants are replacements for the ones originally emitted by the cfg parsing machinery; so these errors are not "new".
- the `InvalidFlag` variant is new, this PR turns this (from being ignored and silently doing nothing) into an error:
```rust
#[rustc_on_unimplemented(on(something, message = "y"))]
//~^ ERROR invalid boolean flag
//~^^ NOTE expected one of `crate_local`, `direct` or `from_desugaring`, not `something`
trait InvalidFlag {}
```
This does not occur anywhere except in this test. I couldn't find a way that I liked to keep allowing this or to do nothing, erroring was the cleanest solution.
- There are a bunch of FIXME throughout this and the previous PR, I plan on addressing those in follow up prs..
Finally, this gets rid of the "longest" dependency in rustc:
Remove backtrace dep from anyhow in features status dump tool
According to `anyhow`'s Cargo.toml:
> On compilers older than 1.65, features=["backtrace"] may be used to enable
> backtraces via the `backtrace` crate. This feature has no effect on 1.65+
> besides bringing in an unused dependency, as `std::backtrace` is always
> preferred.
So this is just bringing in an unused dependency.
This commit does the following:
- Replaces use of rustc_type_ir by rustc_middle in rustc_infer.
- The DelayedMap type is exposed by rustc_middle so everything can be
accessed through rustc_middle in a coherent manner.
- API-layer traits, like InferCtxtLike, Interner or inherent::* must be
accessed via rustc_type_ir, not rustc_middle::ty. For this reason
these are not reexported by rustc_middle::ty.
- Replaces use of ty::Interner by rustc_type_ir::Interner in
rustc_trait_selection
According to `anyhow`'s Cargo.toml:
> On compilers older than 1.65, features=["backtrace"] may be used to enable
> backtraces via the `backtrace` crate. This feature has no effect on 1.65+
> besides bringing in an unused dependency, as `std::backtrace` is always
> preferred.
So this is just bringing in an unused dependency.
This fixes issues with RustAnalyzer not finding stable_mir crate.
It is also part of the long term architecture plan for these crates,
since we are moving towards having stable_mir depend on rustc_smir and
not the other way around.
I believe this is an utility function that will come handy eventually
for stable_mir users, but I'm keeping it as part of rustc_internal since
it initializes the StableMir context and requires `TyCtxt`.
Finally, I added the rustc_internal crate under a feature since the APIs
from this module shall not be stabilized.
This commit does the following:
- Replaces use of rustc_type_ir by rustc_middle
- Removes the rustc_type_ir dependency
- The DelayedSet type is exposed by rustc_middle so everything can be
accessed through rustc_middle in a coherent manner.
Move inline asm check to typeck, properly handle aliases
Pull `InlineAsmCtxt` down to `rustc_hir_typeck`, and instead of using things like `Ty::is_copy`, use the `InferCtxt`-aware methods. To fix https://github.com/rust-lang/trait-system-refactor-initiative/issues/189, we also add a `try_structurally_resolve_*` call to `expr_ty`.
r? lcnr
Update wasm-component-ld to 0.5.13
This commit updates the vendored `wasm-component-ld` binary to 0.5.13 which includes some various bug fixes and new feature updates for upcoming component model features coming down the pike. Not expected to break any existing workflows, just a normal update.
Rollup of 8 pull requests
Successful merges:
- #139865 (Stabilize proc_macro::Span::{start,end,line,column}.)
- #140086 (If creating a temporary directory fails with permission denied then retry with backoff)
- #140216 (Document that "extern blocks must be unsafe" in Rust 2024)
- #140253 (Add XtensaAsmPrinter)
- #140272 (Improve error message for `||` (or) in let chains)
- #140305 (Track per-obligation recursion depth only if there is inference in the new solver)
- #140306 (handle specialization in the new trait solver)
- #140308 (stall generator witness obligations: add regression test)
r? `@ghost`
`@rustbot` modify labels: rollup
If creating a temporary directory fails with permission denied then retry with backoff
On Windows, if creating a temporary directory fails with permission denied then use a retry/backoff loop. This hopefully fixes a recuring error in our CI.
cc ```@jieyouxu,``` https://github.com/rust-lang/rust/issues/133959
[compiletest] Parallelize test discovery
Certain filesystems are slow to service individual read requests, but can service many in parallel. This change brings down the time to run a single cached test on one of those filesystems from 40s to about 8s.
On Windows, if creating a temporary directory fails with permission denied then use a retry/backoff loop. This hopefully fixes a recuring error in our CI.
transmute: Mark edges by byte sets, not byte values
This leads to drastic performance improvements. For example, on the author's 2024 MacBook Pro, the time to convert the `Tree` representation of a `u64` to its equivalent DFA representation drops from ~8.5ms to ~1us, a reduction of ~8,500x. See `bench_dfa_from_tree`.
Similarly, the time to execute a transmutability query from `u64` to `u64` drops from ~35us to ~1.7us, a reduction of ~20x. See `bench_transmute`.
r? `@jswrenn`
This commit updates the vendored `wasm-component-ld` binary to 0.5.13
which includes some various bug fixes and new feature updates for
upcoming component model features coming down the pike. Not expected to
break any existing workflows, just a normal update.
In the `Tree` and `Dfa` representations of a type's layout, store byte
ranges rather than needing to separately store each byte value. This
permits us to, for example, represent a `u8` using a single 0..=255 edge
in the DFA rather than using 256 separate edges.
This leads to drastic performance improvements. For example, on the
author's 2024 MacBook Pro, the time to convert the `Tree` representation
of a `u64` to its equivalent DFA representation drops from ~8.5ms to
~1us, a reduction of ~8,500x. See `bench_dfa_from_tree`.
Similarly, the time to execute a transmutability query from `u64` to
`u64` drops from ~35us to ~1.7us, a reduction of ~20x. See
`bench_transmute`.
Certain filesystems for large monorepos are slow to service individual
read requests, but can service many in parallel. This change brings down
the time to run a single cached test on one of those filesystems from
40s to about 8s.
run-make: drop `os_pipe` workaround now that `anonymous_pipe` is stable on beta
Follow-up to #137537 where I had to include a temporary dep on `os_pipe` before `anonymous_pipe` was stabilized. Now that `anonymous_pipe` is stable on beta, we can get rid of this workaround.
Closes#137532. (Final cleanup item)
r? `@Kobzol`
update ```miniz_oxide``` to 0.8.8
I would normally let the auto actions handle this but it turns out 0.8.7 can trigger a panic when debug assertions are enabled in a few cases so I feel it's important it gets sorted more quickly. (and I would ideally like to yank that version but was worried that could cause some issues had been pulled in as a dependency by this repo already before I discovered the problem)
As it can only happen when debug assertions are enabled (the overflow results in the intended result so it doesn't cause any issue in release mode) and using the wrapping buffer mode when decompressing it is very unlikely to cause any issues here but I would like to get it sorted just to be safe. ```miniz_oxide``` is used in the standard library (and some tools) via ```backtrace-rs ``` which doesn't use a wrapping buffer, and thus won't trigger this condition. There does however seem like there are some tools that do dependency on ```flate2``` which does use ```miniz_oxide``` decompression using a a wrapping buffer and could in theory trigger it if they are run when compiled with debug assertions enabled.
It's kinda unclear what version what tool uses though as several of them specify older versions of flate2 which depended on ```miniz_oxide``` 0.7.x in cargo.toml, and ```miniz_oxide```, and not all have a cargo.lock and due to an older version of ```backtrace``` being in the root Cargo.lock which still depended on ```miniz_oxide``` 0.7.4, so that version is also pulled in alongside the newer version.
Update windows-bindgen to 0.61.0
This updates the automatically generate Windows API bindings. Not much changed this time:
- There's now `Default` implementations for many types, which is convenient. It does however conflict with one place where we implemented a non-zeroed default (to set the length field). But that's no big problem.
- The `--no-core` flag has been renamed to `--no-deps` to more accurately reflect its meaning (i.e. generate all necessary code without requiring additional dependencies).
- The `--link` flag allows us to set the location of the `link!` macro. Currently we use our workspace's `windows_targets` crate but we could move it into library/std using `--link`. However, this would need to be co-ordinated with the `backtrace` crate (which is a separate crate but included in std using `#[path]`). So I've left that for another time.
