This commit is a large change to the implementation of filesystem and
other system-related operations on WASI targets. Previously the standard
library explicitly used the `wasi` crate at the 0.11.x version track
which means that it used WASIp1 APIs directly. This meant that `std` was
hard-coded to use WASIp1 syscalls and there was no separate
implementation for the WASIp{2,3} targets, for example. The high-level
goal of this commit is to decouple this interaction and avoid the use of
the `wasi` crate on the WASIp2 target.
Historically when WASIp1 was originally added to Rust the wasi-libc
library was in a much different position than it is today. Nowadays Rust
already depends on wasi-libc on WASI targets for things like memory
allocation and environment variable management. As a libc library it
also has all the functions necessary to implement all filesystem
operations Rust wants. Recently wasi-libc additionally was updated to
use WASIp2 APIs directly on the `wasm32-wasip2` target instead of using
`wasm32-wasip1` APIs. This commit is leveraging this work by enabling
Rust to completely sever the dependence on WASIp1 APIs when compiling
for `wasm32-wasip2`. This is also intended to make it easier to migrate
to `wasm32-wasip3` internally in the future where now only libc need be
updated and Rust doesn't need to explicitly change as well.
The overall premise of this commit is that there's no need for
WASI-specific implementation modules throughout the standard library.
Instead the libc-style bindings already implemented for Unix-like
targets are sufficient. This means that Rust will now be using
libc-style interfaces to interact with the filesystem, for example, and
wasi-libc is the one responsible for translating these POSIX-ish
functions into WASIp{1,2} calls.
Concrete changes here are:
* `std` for `wasm32-wasip2` no longer depends on `wasi 0.11.x`
* The implementation of `std::os::wasi::fs`, which was previously
unstable and still is, now has portions gated to only work on the
WASIp1 target which use the `wasi` crate directly. Traits have been
trimmed down in some cases, updated in others, or now present a
different API on WASIp1 and WASIp2. It's expected this'll get further
cleanup in the future.
* The `std::sys::fd::wasi` module is deleted and `unix` is used instead.
* The `std::sys::fs::wasi` module is deleted and `unix` is used instead.
* The `std::sys::io::io_slice::wasi` module is deleted and `unix` is used
instead.
* The `std::sys::pal::{wasip1,wasip2}` modules are now merged together
as their difference is much smaller than before.
* The `std::sys::pal::wasi::time` is deleted and the `unix` variant is
used directly instead.
* The `std::sys::stdio::wasip{1,2}` modules are deleted and the `unix`
variant is used instead.
* The `std::sys:🧵:wasip{1,2}` modules are deleted and the `unix`
variant is used instead.
Overall Rust's libstd is effectively more tightly bound to libc when
compiled to WASI targets. This is intended to mirror how it's expected
all other languages will also bind to WASI. This additionally has the
nice goal of drastically reducing the WASI-specific maintenance burden
in libstd (in theory) and the only real changes required here are extra
definitions being added to `libc` (done in separate PRs). This might be
required for more symbols in the future but for now everything should be
mostly complete.
Update memchr to 2.7.6
memchr 2.7.6 contains a bugfix for aarch64_be.
Note: I'm not sure about how dependency updates are managed in rust.git. If something should go through another branch or will happen automatically, please let me know.
std: Add Motor OS std library port
Motor OS was added as a no-std Tier-3 target in
[PR 146848](https://github.com/rust-lang/rust/pull/146848) as x86_64-unknown-motor.
This PR adds the std library for Motor OS.
While the PR may seem large, all it does is proxy
std pal calls to [moto-rt](https://crates.io/crates/moto-rt). Where there is some non-trivial
code (e.g. thread::spawn), it is quite similar, often
identical, to what other platforms do.
Motor OS was added as a no-std Tier-3 target in
https://github.com/rust-lang/rust/pull/146848
as x86_64-unknown-motor.
This patch/PR adds the std library for Motor OS.
While the patch may seem large, all it does is proxy
std pal calls to moto-rt. When there is some non-trivial
code (e.g. thread::spawn), it is quite similar, and often
identical, to what other platforms do.
The recent work on the WASIp2 target being integrated into the standard
library ended up causing a bug in nightly on the target. This has now
been fixed in the `wasi` crate so this commit pulls in the updated
version to ensure bindings work correctly.
This commit is the start of an effort to support WASIp2 natively in the
standard library. Before this commit the `wasm32-wasip2` target behaved
exactly like `wasm32-wasip1` target by importing APIs from the core wasm
module `wasi_snapshot_preview1`. These APIs are satisfied by the
`wasm-component-ld` target by using an [adapter] which implements WASIp1
in terms of WASIp2. This adapter comes at a cost, however, in terms of
runtime indirection and instantiation cost, so ideally the adapter would
be removed entirely. The purpose of this adapter was to provide a
smoother on-ramp from WASIp1 to WASIp2 when it was originally created.
The `wasm32-wasip2` target has been around for long enough now that it's
much more established. Additionally the only thing historically blocking
using WASIp2 directly was implementation effort. Work is now underway to
migrate wasi-libc itself to using WASIp2 directly and now seems as good
a time as any to migrate the Rust standard library too.
Implementation-wise the milestones here are:
* The `wasm32-wasip2` target now also depends on the `wasi` crate at
version 0.14.* in addition to the preexisting dependency of 0.11.*.
The 0.14.* release series binds WASIp2 APIs instead of WASIp1 APIs.
* Some preexisting naming around `mod wasi` or `wasi.rs` was renamed to
`wasip1` where appropriate. For example `std::sys::pal::wasi` is now
called `std::sys::pal::wasip1`.
* More platform-specific WASI modules are now split between WASIp1 and
WASIp2. For example getting the current time, randomness, and
process arguments now use WASIp2 APIs directly instead of using WASIp1
APIs that require an adapter.
It's worth pointing out that this PR does not migrate the entire
standard library away from using WASIp1 APIs on the `wasm32-wasip2`
target. Everything related to file descriptors and filesystem APIs is
still using WASIp1. Migrating that is left for a future PR. In the
meantime the goal of this change is to lay the groundwork necessary for
migrating in the future. Eventually the goal is to drop the `wasi`
0.11.* dependency on the `wasm32-wasip2` target (the `wasm32-wasip1`
target will continue to retain this dependency).
[adapter]: https://github.com/bytecodealliance/wasmtime/blob/main/crates/wasi-preview1-component-adapter/README.md
library: Migrate from `cfg_if` to `cfg_select`
Migrate the standard library from using the external `cfg_if` crate to using the now-built-in `cfg_select` macro.
This does not yet eliminate the dependency from `library/std/Cargo.toml`, because while the standard library itself no longer uses `cfg_if`, it also incorporates the `backtrace` crate, which does.
Migration assisted by the following vim command (after selecting the full `cfg_if!` invocation):
```
'<,'>s/\(cfg_if::\)\?cfg_if/cfg_select/ | '<,'>s/^\( *\)} else {/\1}\r\1_ => {/c | '<,'>s/^\( *\)} else if #\[cfg(\(.*\))\] /\1}\r\1\2 => /e | '<,'>s/if #\[cfg(\(.*\))\] {/\1 => {/e
```
This is imperfect, but substantially accelerated the process. This prompts for confirmation on the `} else {` since that can also appear inside one of the arms. This also requires manual intervention to handle any multi-line conditions.
Migrate the standard library from using the external `cfg_if` crate to
using the now-built-in `cfg_select` macro.
This does not yet eliminate the dependency from
`library/std/Cargo.toml`, because while the standard library itself no
longer uses `cfg_if`, it also incorporates the `backtrace` crate, which
does.
Migration assisted by the following vim command (after selecting the
full `cfg_if!` invocation):
```
'<,'>s/\(cfg_if::\)\?cfg_if/cfg_select/ | '<,'>s/^\( *\)} else {/\1}\r\1_ => {/c | '<,'>s/^\( *\)} else if #\[cfg(\(.*\))\] /\1}\r\1\2 => /e | '<,'>s/if #\[cfg(\(.*\))\] {/\1 => {/e
```
This is imperfect, but substantially accelerated the process. This
prompts for confirmation on the `} else {` since that can also appear
inside one of the arms. This also requires manual intervention to handle
any multi-line conditions.
`compiler-builtins` is already in the crate graph via `alloc`, and all
features related to `compiler-builtins` goes through `alloc`. There
isn't any reason that `std` needs this direct dependency, so remove it.
The three panic-related library crates need to have access to `core`,
and `compiler-builtins` needs to be in the crate graph. Rather than
specifying both dependencies, switch these crates to use
`rustc-std-workspace-core` which already does this.
This means there is now a single place that the `compiler-builtins`
dependency needs to get configured, for everything other than `alloc`
and `std`.
Upgrade the `fortanix-sgx-abi` dependency
0.6.1 removes the `compiler-builtins` dependency, part of RUST-142265. The breaking change from 0.5 to 0.6 is for an update to the `insecure_time` API [1].
I validated that `./x c library --target x86_64-fortanix-unknown-sgx` completes successfully with this change.
Link: a34e9767f3 [1]
This primarily pulls in alexcrichton/dlmalloc-rs/55 and
alexcrichton/dlmalloc-rs/54 to address 144199. Notably the highest byte
in the wasm address space is no longer allocatable and additionally the
allocator internally uses `wrapping_add` instead of `add` on pointers
since on 32-bit platforms offsets might be larger than half the address
space.
0.6.1 removes the `compiler-builtins` dependency, part of RUST-142265.
The breaking change from 0.5 to 0.6 is for an update to the
`insecure_time` API [1].
I validated that `./x c library --target x86_64-fortanix-unknown-sgx`
completes successfully with this change.
Link: a34e9767f3 [1]
update to literal-escaper 0.0.4 for better API without `unreachable` and faster string parsing
This is the replacement for just the part of https://github.com/rust-lang/rust/pull/138163 dealing with the changed API of unescape functionality, since that got moved into its own crate.
<del>This uses an unpublished version of literal-escaper (https://github.com/rust-lang/literal-escaper/pull/8).</del>
r? `@nnethercote`
Upgrade `object`, `addr2line`, and `unwinding` in the standard library
Object:
0.37.0 is a semver-breaking release but the only breakage is in `elf::R_RISCV_GNU_*` and `pe::IMAGE_WEAK_EXTERN_*` constants, as well as Mach-O dyld. This API is not used by `std`, so we should be fine to upgrade.
This new version also includes functionality for parsing Wasm object files that we may eventually like to make use of.
Changelog: https://github.com/gimli-rs/object/blob/master/CHANGELOG.md#0370
Addr2line:
0.25.0 is a breaking change only because it upgrades the `gimli` version. It also includes a change to the `compiler-builtins` dependency that helps with [1].
Changelog: https://github.com/gimli-rs/addr2line/blob/master/CHANGELOG.md#0250-20250611
[1]: https://github.com/rust-lang/rust/issues/142265
Update dependencies in `library/Cargo.lock`
This removes the `compiler_builtins` dependency from a handful of library dependencies, which is progress toward [1].
[1]: https://github.com/rust-lang/rust/issues/142265
0.37.0 is a semver-breaking release but the only breakage is in
`elf::R_RISCV_GNU_*` and `pe::IMAGE_WEAK_EXTERN_*` constants, as well as
Mach-O dyld. This API is not used by `std`, so we should be fine to
upgrade.
This new version also includes functionality for parsing Wasm object
files that we may eventually like to make use of.
Also includes the minor bump from 0.37.0 to 0.37.1 to help [1].
Changelog: https://github.com/gimli-rs/object/blob/master/CHANGELOG.md#0370
[1]: https://github.com/rust-lang/rust/issues/142265
