Win: Use POSIX rename semantics for `std::fs::rename` if available
Windows 10 1601 introduced `FileRenameInfoEx` as well as `FILE_RENAME_FLAG_POSIX_SEMANTICS`, allowing for atomic renaming and renaming if the target file is has already been opened with `FILE_SHARE_DELETE`, in which case the file gets renamed on disk while the open file handle still refers to the old file, just like in POSIX. This resolves#123985, where atomic renaming proved difficult to impossible due to race conditions.
If `FileRenameInfoEx` isn't available due to missing support from the underlying filesystem or missing OS support, the renaming is retried with `FileRenameInfo`, which matches the behavior of `MoveFileEx`.
This PR also manually replicates parts of `MoveFileEx`'s internal logic, as reverse-engineered from the disassembly: If the source file is a reparse point and said reparse point is a mount point, the mount point itself gets renamed; otherwise the reparse point is resolved and the result renamed.
Notes:
- Currently, the `win7` target doesn't bother with `FileRenameInfoEx` at all; it's probably desirable to remove that special casing and try `FileRenameInfoEx` anyway if it doesn't exist, in case the binary is run on newer OS versions.
Fixes#123985
Abstract `ProcThreadAttributeList` into its own struct
As extensively discussed in issue #114854, the current implementation of the unstable `windows_process_extensions_raw_attribute` features lacks support for passing a raw pointer.
This PR wants to explore the opportunity to abstract away the `ProcThreadAttributeList` into its own struct to for one improve safety and usability and secondly make it possible to maybe also use it to spawn new threads.
try-job: x86_64-mingw
Field init shorthand allows writing initializers like `tcx: tcx` as
`tcx`. The compiler already uses it extensively. Fix the last few places
where it isn't yet used.
Run TLS destructors for wasm32-wasip1-threads
The target wasm32-wasip1-threads has support for pthreads and allows registration of TLS destructors.
For spawned threads, this registers Rust TLS destructors by creating a pthreads key with an attached destructor function.
For the main thread, this registers an `atexit` handler to run the TLS destructors.
try-job: test-various
wasi/fs: Improve stopping condition for <ReadDir as Iterator>::next
When upgrading [Zed](https://github.com/zed-industries/zed/pull/19349) to Rust 1.82 I've encountered a test failure in our test suite. Specifically, one of our extension tests started hanging. I've tracked it down to a call to std::fs::remove_dir_all not returning when an extension is compiled with Rust 1.82 Our extension system uses WASM components, thus I've looked at the diff between 1.81 and 1.82 with respect to WASI and found 736f773844
As it turned out, calling remove_dir_all from extension returned io::ErrorKind::NotFound in 1.81; the underlying issue is that the ReadDir iterator never actually terminates iteration, however since it loops around, with 1.81 we'd come across an entry second time and fail to remove it, since it would've been removed previously. With 1.82 and 736f773844 it is no longer the case, thus we're seeing the hang. The tests do pass when everything but the extensions is compiled with 1.82.
This commit makes ReadDir::next adhere to readdir contract, namely it will no longer call readdir once the returned # of bytes is smaller than the size of a passed-in buffer. Previously we'd only terminate the loop if readdir returned 0.
Improve comments for the default backtrace printer
The existing comments were misleading, confusing, and outdated.
Take this comment for example:
```
// Any frames between `__rust_begin_short_backtrace` and `__rust_end_short_backtrace`
// are omitted from the backtrace in short mode, `__rust_end_short_backtrace` will be
// called before the panic hook, so we won't ignore any frames if there is no
// invoke of `__rust_begin_short_backtrace`.
```
this is just wrong. here is an example (full) backtrace:
<details>
```
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
Running `/home/jyn/.local/lib/cargo/target/debug/example`
called `Option::unwrap()` on a `None` value
stack backtrace:
0: 0x56499698c595 - std::backtrace_rs::backtrace::libunwind::trace::h5ef2cc16e9a7415a
1: 0x56499698c595 - std::backtrace_rs::backtrace::trace_unsynchronized::h9b5e016e9075f714
2: 0x56499698c595 - std::sys_common::backtrace::_print_fmt::h2f62c7f9ff224e93
3: 0x56499698c595 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hbe51682735731910
4: 0x5649969aa26b - core::fmt::rt::Argument::fmt::h1994ab2b310d665e
5: 0x5649969aa26b - core::fmt::write::hade58a36d63468d7
6: 0x56499698a43f - std::io::Write::write_fmt::h16145587d801a9ab
7: 0x56499698c36e - std::sys_common::backtrace::_print::ha8082e56201dadb4
8: 0x56499698c36e - std::sys_common::backtrace::print::he30f96b4e7f6cbfd
9: 0x56499698d709 - std::panicking::default_hook::{{closure}}::hf0801f6b18a968d3
10: 0x56499698d4ac - std::panicking::default_hook::hd2defec7eda5aeb0
11: 0x56499698dc31 - std::panicking::rust_panic_with_hook::hde93283600065c53
12: 0x56499698daf3 - std::panicking::begin_panic_handler::{{closure}}::h5e151adbdb7ec0c1
13: 0x56499698ca59 - std::sys_common::backtrace::__rust_end_short_backtrace::he36a1407e0f77700
14: 0x56499698d7d4 - rust_begin_unwind
15: 0x5649969a9503 - core::panicking::panic_fmt::h2380d41365f95412
16: 0x5649969a958c - core::panicking::panic::h38cf8db80e8c6e67
17: 0x5649969a93e9 - core::option::unwrap_failed::he72696e53ff29a05
18: 0x5649969722b6 - core::option::Option<T>::unwrap::hb574dc0dc1703062
19: 0x5649969722b6 - example::main::h7a867aafacd93d75
20: 0x5649969721db - core::ops::function::FnOnce::call_once::h734f99a5e57291b7
21: 0x56499697226e - std::sys_common::backtrace::__rust_begin_short_backtrace::h02f5d58c351c4756
22: 0x564996972241 - std::rt::lang_start::{{closure}}::h8b134fe2c31a4355
23: 0x564996988662 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h88d7bb571ee2aaf4
24: 0x564996988662 - std::panicking::try::do_call::hfb78dfb6599c871d
25: 0x564996988662 - std::panicking::try::habd041c8c4c8e50c
27: 0x564996988662 - std::rt::lang_start_internal::{{closure}}::h227591a6f9c0879e
28: 0x564996988662 - std::panicking::try::do_call::h3c5878333c38916a
29: 0x564996988662 - std::panicking::try::h5af7b3a127cdae70
31: 0x564996988662 - std::rt::lang_start_internal::hbc85e809eeace0dd
32: 0x56499697221a - std::rt::lang_start::ha1eb16922c9cb224
33: 0x5649969722ee - main
34: 0x7f031962a1ca - __libc_start_call_main
35: 0x7f031962a28b - __libc_start_main_impl
36: 0x5649969720a5 - _start
37: 0x0 - <unknown>
```
</details>
note particularly frames 13-21, from start_backtrace to end_backtrace. with PrintFmt::Short, these are the *only* frames that are printed; i.e. we are doing the exact opposite of the comment.
r? ``@saethlin``
The existing comments were misleading, confusing, and wrong.
Take this comment for example:
```
// Any frames between `__rust_begin_short_backtrace` and `__rust_end_short_backtrace`
// are omitted from the backtrace in short mode, `__rust_end_short_backtrace` will be
// called before the panic hook, so we won't ignore any frames if there is no
// invoke of `__rust_begin_short_backtrace`.
```
this is just wrong. here is an example (full) backtrace:
```
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
Running `/home/jyn/.local/lib/cargo/target/debug/example`
called `Option::unwrap()` on a `None` value
stack backtrace:
0: 0x56499698c595 - std::backtrace_rs::backtrace::libunwind::trace::h5ef2cc16e9a7415a
1: 0x56499698c595 - std::backtrace_rs::backtrace::trace_unsynchronized::h9b5e016e9075f714
2: 0x56499698c595 - std::sys_common::backtrace::_print_fmt::h2f62c7f9ff224e93
3: 0x56499698c595 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hbe51682735731910
4: 0x5649969aa26b - core::fmt::rt::Argument::fmt::h1994ab2b310d665e
5: 0x5649969aa26b - core::fmt::write::hade58a36d63468d7
6: 0x56499698a43f - std::io::Write::write_fmt::h16145587d801a9ab
7: 0x56499698c36e - std::sys_common::backtrace::_print::ha8082e56201dadb4
8: 0x56499698c36e - std::sys_common::backtrace::print::he30f96b4e7f6cbfd
9: 0x56499698d709 - std::panicking::default_hook::{{closure}}::hf0801f6b18a968d3
10: 0x56499698d4ac - std::panicking::default_hook::hd2defec7eda5aeb0
11: 0x56499698dc31 - std::panicking::rust_panic_with_hook::hde93283600065c53
12: 0x56499698daf3 - std::panicking::begin_panic_handler::{{closure}}::h5e151adbdb7ec0c1
13: 0x56499698ca59 - std::sys_common::backtrace::__rust_end_short_backtrace::he36a1407e0f77700
14: 0x56499698d7d4 - rust_begin_unwind
15: 0x5649969a9503 - core::panicking::panic_fmt::h2380d41365f95412
16: 0x5649969a958c - core::panicking::panic::h38cf8db80e8c6e67
17: 0x5649969a93e9 - core::option::unwrap_failed::he72696e53ff29a05
18: 0x5649969722b6 - core::option::Option<T>::unwrap::hb574dc0dc1703062
19: 0x5649969722b6 - example::main::h7a867aafacd93d75
20: 0x5649969721db - core::ops::function::FnOnce::call_once::h734f99a5e57291b7
21: 0x56499697226e - std::sys_common::backtrace::__rust_begin_short_backtrace::h02f5d58c351c4756
22: 0x564996972241 - std::rt::lang_start::{{closure}}::h8b134fe2c31a4355
23: 0x564996988662 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h88d7bb571ee2aaf4
24: 0x564996988662 - std::panicking::try::do_call::hfb78dfb6599c871d
25: 0x564996988662 - std::panicking::try::habd041c8c4c8e50c
27: 0x564996988662 - std::rt::lang_start_internal::{{closure}}::h227591a6f9c0879e
28: 0x564996988662 - std::panicking::try::do_call::h3c5878333c38916a
29: 0x564996988662 - std::panicking::try::h5af7b3a127cdae70
31: 0x564996988662 - std::rt::lang_start_internal::hbc85e809eeace0dd
32: 0x56499697221a - std::rt::lang_start::ha1eb16922c9cb224
33: 0x5649969722ee - main
34: 0x7f031962a1ca - __libc_start_call_main
35: 0x7f031962a28b - __libc_start_main_impl
36: 0x5649969720a5 - _start
37: 0x0 - <unknown>
```
note particularly frames 13-21, from start_backtrace to end_backtrace. with PrintFmt::Short, these are the *only* frames that are printed; i.e. we are doing the exact opposite of the comment.
fix: hurd build, stat64.st_fsid was renamed to st_dev
On hurd, `stat64.st_fsid` was renamed to `st_dev` in https://github.com/rust-lang/libc/pull/3785, so if you have a new libc with this patch included, and you build std from source, you get this error:
```sh
error[E0609]: no field `st_fsid` on type `&stat64`
--> /home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/os/hurd/fs.rs:301:36
|
301 | self.as_inner().as_inner().st_fsid as u64
| ^^^^^^^ unknown field
|
help: a field with a similar name exists
|
301 | self.as_inner().as_inner().st_uid as u64
| ~~~~~~
```
Full CI log: https://github.com/nix-rust/nix/actions/runs/12033180710/job/33546728266?pr=2544
std: refactor `pthread`-based synchronization
The non-trivial code for `pthread_condvar` is duplicated across the thread parking and the `Mutex`/`Condvar` implementations. This PR moves that code into `sys::pal`, which now exposes an `unsafe` wrapper type for `pthread_mutex_t` and `pthread_condvar_t`.
thread::available_parallelism for wasm32-wasip1-threads
The target has limited POSIX support and provides the `libc::sysconf` function which allows querying the number of available CPUs.
Fix and undeprecate home_dir()
`home_dir()` has been deprecated for 6 years due to using `HOME` env var on Windows.
It's been a long time, and having a perpetually buggy and deprecated function in the standard library is not useful. I propose fixing and undeprecating it.
6 years seems more than long enough to warn users against relying on this function. The change in behavior is minor, and it's more of a bug fix than breakage. The old behavior is unlikely to be useful, and even if anybody actually needed to specifically use the non-standard `HOME` on Windows, they can trivially mitigate this change by reading the env var themselves.
----
Use of `USERPROFILE` is in line with the `home` crate: 37bc5f0232/crates/home/src/windows.rs (L12)
The `home` crate uses `SHGetKnownFolderPath` instead of `GetUserProfileDirectoryW`. AFAIK it doesn't make any difference in practice, because `SHGetKnownFolderPath` merely adds support for more kinds of folders, including virtual (non-filesystem) folders identified by a GUID, but the specific case of [`FOLDERID_Profile`](https://learn.microsoft.com/en-us/windows/win32/shell/knownfolderid#FOLDERID_Profile) is documented as a FIXED folder (a regular filesystem path). Just in case, I've added a note to documentation that the use of `GetUserProfileDirectoryW` can change.
I've used `CURRENT_RUSTC_VERSION` in a doccomment. `replace-version-placeholder` tool seems to perform a simple string replacement, so hopefully it'll get updated.
Bump boostrap compiler to new beta
Currently failing due to something about the const stability checks and `panic!`. I'm not sure why though since I wasn't able to see any PRs merged in the past few days that would result in a `cfg(bootstrap)` that shouldn't be removed. cc `@RalfJung` #131349
[AIX] create shim for lgammaf_r
On AIX, we don't have 32bit floating point for re-entrant `lgammaf_r` but we do have the 64bit floating point re-entrant `lgamma_r` so we can use the 64bit version instead and truncate back to a 32bit float.
This solves the linker missing symbol for `.lgammaf_r` when testing and using these parts of the `std`.
std: allow after-main use of synchronization primitives
By creating an unnamed thread handle when the actual one has already been destroyed, synchronization primitives using thread parking can be used even outside the Rust runtime.
This also fixes an inefficiency in the queue-based `RwLock`: if `thread::current` was not initialized yet, it will create a new handle on every parking attempt without initializing `thread::current`. The private `current_or_unnamed` function introduced here fixes this.
When upgrading [Zed](https://github.com/zed-industries/zed/pull/19349) to Rust 1.82 I've encountered a test failure in our test suite. Specifically, one of our extension tests started hanging. I've tracked it down to a call to std::fs::remove_dir_all not returning when an extension is compiled with Rust 1.82
Our extension system uses WASM components, thus I've looked at the diff between 1.81 and 1.82 with respect to WASI and found 736f773844
As it turned out, calling remove_dir_all from extension returned io::ErrorKind::NotFound in 1.81;
the underlying issue is that the ReadDir iterator never actually terminates iteration, however since it loops around, with 1.81 we'd come across an entry second time and fail to remove it, since it would've been removed previously.
With 1.82 and 736f773844 it is no longer the case, thus we're seeing the hang.
This commit makes ReadDir::next adhere to readdir contract, namely it will no longer call readdir once the returned # of bytes is smaller than the size of a passed-in buffer.
Previously we'd only terminate the loop if readdir returned 0.
By creating an unnamed thread handle when the actual one has already been destroyed, synchronization primitives using thread parking can be used even outside the Rust runtime.
This also fixes an inefficiency in the queue-based `RwLock`: if `thread::current` was not initialized yet, it will create a new handle on every parking attempt without initializing `thread::current`. The private `current_or_unnamed` function introduced here fixes this.
Rwlock downgrade
Tracking Issue: #128203
This PR adds a `downgrade` method for `RwLock` / `RwLockWriteGuard` on all currently supported platforms.
Outstanding questions:
- [x] ~~Does the `futex.rs` change affect performance at all? It doesn't seem like it will but we can't be certain until we bench it...~~
- [x] ~~Should the SOLID platform implementation [be ported over](https://github.com/rust-lang/rust/pull/128219#discussion_r1693470090) to the `queue.rs` implementation to allow it to support downgrades?~~
This commit fixes a memory ordering bug in the futex implementation
(`Relaxed` -> `Release` on `downgrade`).
This commit also removes a badly written test that deadlocked and
replaces it with a more reasonable test based on an already-tested
`downgrade` test from the parking-lot crate.
This commit adds the `downgrade` method onto the inner `RwLock` queue
implementation.
There are also a few other style patches included in this commit.
Co-authored-by: Jonas Böttiger <jonasboettiger@icloud.com>
This commit only has documentation changes and a few things moved around
the file. The very few code changes are cosmetic: changes like turning a
`match` statement into an `if let` statement or reducing indentation for
long if statements.
This commit also adds several safety comments on top of `unsafe` blocks
that might not be immediately obvious to a first-time reader.
Code "changes" are in:
- `add_backlinks_and_find_tail`
- `lock_contended`
A majority of the changes are just expanding the comments from 80
columns to 100 columns.
