This commit rejiggers the generics used in the implementation of
`Command::env` with the purpose of reducing the amount of codegen that
needs to happen in consumer crates, instead preferring to generate code
into libstd.
This was found when profiling the compile times of the `cc` crate where
the binary rlib produced had a lot of `BTreeMap` code compiled into it
but the crate doesn't actually use `BTreeMap`. It turns out that
`Command::env` is generic enough to codegen the entire implementation in
calling crates, but in this case there's no performance concern so it's
fine to compile the code into the standard library.
This change is done by removing the generic on the `CommandEnv` map
which is intended to handle case-insensitive variables on Windows.
Instead now a generic isn't used but rather a `use` statement defined
per-platform is used.
With this commit a debug build of `Command::new("foo").env("a", "b")`
drops from 21k lines of LLVM IR to 10k.
Attempt to create sockets with the WSA_FLAG_NO_HANDLE_INHERIT flag, and
handle the potential error gracefully (as the flag isn't support on
Windows 7 before SP1)
On Windows process exit codes are never signals but rather always 32-bit
integers. Most faults like segfaults and such end up having large
integers used to represent them, like STATUS_ACCESS_VIOLATION being
0xC0000005. Currently, however, when an `ExitStatus` is printed this
ends up getting rendered as 3221225477 which is somewhat more difficult
to debug.
This commit adds a branch in `Display for ExitStatus` on Windows which
handles exit statuses where the high bit is set and prints those exit
statuses as hex instead of with decimals. This will hopefully preserve
the current display for small exit statuses (like `exit code: 22`), but
assist in quickly debugging segfaults/access violations/etc. I've
found at least that the hex codes are easier to search for than decimal.
I wasn't able to find any official documentation saying that all system
exit codes have the high bit set, but I figure it's a good enough
heuristic for now.
These were showing up in tests and in binaries but are trivially optimize-able
away, so add `#[inline]` attributes so LLVM has an opportunity to optimize them
out.
`Stdio` now implements `From<ChildStdin>`, `From<ChildStdout>`,
`From<ChildStderr>`, and `From<File>`.
The `Command::stdin`/`stdout`/`stderr` methods now take any type that
implements `Into<Stdio>`.
This makes it much easier to write shell-like command chains, piping to
one another and redirecting to and from files. Otherwise one would need
to use the unsafe and OS-specific `from_raw_fd` or `from_raw_handle`.
`CreateProcess` will interpret args as part of the binary name if it
doesn't find the binary using just the unquoted name. For example if
`foo.exe` doesn't exist, `Command::new("foo").arg("bar").spawn()` will
try to launch `foo bar.exe` which is clearly not desired.
This alters the stdio code on Windows to always call `GetStdHandle` whenever the
stdio read/write functions are called as this allows us to track changes to the
value over time (such as if a process calls `SetStdHandle` while it's running).
Closes#40490
This is much nicer for callers who want to short-circuit real I/O errors
with `?`, because they can write this
if let Some(status) = foo.try_wait()? {
...
} else {
...
}
instead of this
match foo.try_wait() {
Ok(status) => {
...
}
Err(err) if err.kind() == io::ErrorKind::WouldBlock => {
...
}
Err(err) => return Err(err),
}
The original design of `try_wait` was patterned after the `Read` and
`Write` traits, which support both blocking and non-blocking
implementations in a single API. But since `try_wait` is never blocking,
it makes sense to optimize for the non-blocking case.
Tracking issue: https://github.com/rust-lang/rust/issues/38903
std: Add a nonblocking `Child::try_wait` method
This commit adds a new method to the `Child` type in the `std::process` module
called `try_wait`. This method is the same as `wait` except that it will not
block the calling thread and instead only attempt to collect the exit status. On
Unix this means that we call `waitpid` with the `WNOHANG` flag and on Windows it
just means that we pass a 0 timeout to `WaitForSingleObject`.
Currently it's possible to build this method out of tree, but it's unfortunately
tricky to do so. Specifically on Unix you essentially lose ownership of the pid
for the process once a call to `waitpid` has succeeded. Although `Child` tracks
this state internally to be resilient to multiple calls to `wait` or a `kill`
after a successful wait, if the child is waited on externally then the state
inside of `Child` is not updated. This means that external implementations of
this method must be extra careful to essentially not use a `Child`'s methods
after a call to `waitpid` has succeeded (even in a nonblocking fashion).
By adding this functionality to the standard library it should help canonicalize
these external implementations and ensure they can continue to robustly reuse
the `Child` type from the standard library without worrying about pid ownership.
This commit adds a new method to the `Child` type in the `std::process` module
called `try_wait`. This method is the same as `wait` except that it will not
block the calling thread and instead only attempt to collect the exit status. On
Unix this means that we call `waitpid` with the `WNOHANG` flag and on Windows it
just means that we pass a 0 timeout to `WaitForSingleObject`.
Currently it's possible to build this method out of tree, but it's unfortunately
tricky to do so. Specifically on Unix you essentially lose ownership of the pid
for the process once a call to `waitpid` has succeeded. Although `Child` tracks
this state internally to be resilient to multiple calls to `wait` or a `kill`
after a successful wait, if the child is waited on externally then the state
inside of `Child` is not updated. This means that external implementations of
this method must be extra careful to essentially not use a `Child`'s methods
after a call to `waitpid` has succeeded (even in a nonblocking fashion).
By adding this functionality to the standard library it should help canonicalize
these external implementations and ensure they can continue to robustly reuse
the `Child` type from the standard library without worrying about pid ownership.
This commit fixes a mistake introduced in #31618 where overlapped handles were
leaked to child processes on Windows. On Windows once a handle is in overlapped
mode it should always have I/O executed with an instance of `OVERLAPPED`. Most
child processes, however, are not prepared to have their stdio handles in
overlapped mode as they don't use `OVERLAPPED` on reads/writes to the handle.
Now we haven't had any odd behavior in Rust up to this point, and the original
bug was introduced almost a year ago. I believe this is because it turns out
that if you *don't* pass an `OVERLAPPED` then the system will [supply one for
you][link]. In this case everything will go awry if you concurrently operate on
the handle. In Rust, however, the stdio handles are always locked, and there's
no way to not use them unlocked in libstd. Due to that change we've always had
synchronized access to these handles, which means that Rust programs typically
"just work".
Conversely, though, this commit fixes the test case included, which exhibits
behavior that other programs Rust spawns may attempt to execute. Namely, the
stdio handles may be concurrently used and having them in overlapped mode wreaks
havoc.
[link]: https://blogs.msdn.microsoft.com/oldnewthing/20121012-00/?p=6343Closes#38811
This adds a CommandExt trait for Windows along with an implementation of it
for std::process::Command with methods to set the process creation flags that
are passed to CreateProcess.
Sometimes a process may be waited on externally from the standard library, in
which case it can be useful to create a raw `ExitStatus` structure to return.
This commit extends the existing Unix `ExitStatusExt` extension trait and adds a
new Windows-specific `ExitStatusExt` extension trait to do this. The methods are
currently called `ExitStatus::from_raw`.
cc #32713
Automated conversion using the untry tool [1] and the following command:
```
$ find -name '*.rs' -type f | xargs untry
```
at the root of the Rust repo.
[1]: https://github.com/japaric/untry
For example if `Command::output` or `Command::status` is used then stdin is just
immediately closed. Add an option for this so as an optimization we can avoid
creating pipes entirely.
This should help reduce the number of active file descriptors when spawning
processes on Unix and the number of active handles on Windows.
Most of this is platform-specific anyway, and we generally have to jump through
fewer hoops to do the equivalent operation on Windows. One benefit for Windows
today is that this new structure avoids an extra `DuplicateHandle` when creating
pipes. For Unix, however, the behavior should be the same.
Note that this is just a pure refactoring, no functionality was added or
removed.
The function `CreateProcess` is not itself unsafe to call from many threads, the
article in question is pointing out that handles can be inherited by unintended
child processes. This is basically the same race as the standard Unix
open-then-set-cloexec race.
Since the intention of the lock is to protect children from inheriting
unintended handles, the lock is now lifted out to before the creation of the
child I/O handles (which will all be inheritable). This will ensure that we only
have one process in Rust at least creating inheritable handles at a time,
preventing unintended inheritance to children.
On Unix we have to be careful to not call `waitpid` twice, but we don't have to
be careful on Windows due to the way process handles work there. As a result the
cached `Option<ExitStatus>` is only necessary on Unix, and it's also just an
implementation detail of the Unix module.
At the same time. also update some code in `kill` on Unix to avoid a wonky
waitpid with WNOHANG. This was added in 0e190b9a to solve #13124, but the
`signal(0)` method is not supported any more so there's no need to for this
workaround. I believe that this is no longer necessary as it's not really doing
anything.
* Build up the argp/envp pointers while the `Command` is being constructed
rather than only when `spawn` is called. This will allow better sharing of
code between fork/exec paths.
* Rename `child_after_fork` to `exec` and have it only perform the exec half of
the spawning. This also means the return type has changed to `io::Error`
rather than `!` to represent errors that happen.
This commit removes the `-D warnings` flag being passed through the makefiles to
all crates to instead be a crate attribute. We want these attributes always
applied for all our standard builds, and this is more amenable to Cargo-based
builds as well.
Note that all `deny(warnings)` attributes are gated with a `cfg(stage0)`
attribute currently to match the same semantics we have today
On all platforms, reading from stdin where the actual stdin isn't present should
return 0 bytes as having been read rather than the entire buffer.
On Windows, handle the case where we're inheriting stdio handles but one of them
isn't present. Currently the behavior is to fail returning an I/O error but
instead this commit corrects it to detecting this situation and propagating the
non-set handle.
Closes#31167
* Delete `sys::unix::{c, sync}` as these are now all folded into libc itself
* Update all references to use `libc` as a result.
* Update all references to the new flat namespace.
* Moves all windows bindings into sys::c
* Store the native representation directly in the `ExitStatus` structure instead
of a "parsed version" (mostly for Unix).
* On Windows, be more robust against processes exiting with the status of 259.
Unfortunately this exit code corresponds to `STILL_ACTIVE`, causing libstd to
think the process was still alive, causing an infinite loop. Instead the loop
is removed altogether and `WaitForSingleObject` is used to wait for the
process to exit.
This commit is an implementation of [RFC 1174][rfc] which adds three new traits
to the standard library:
* `IntoRawFd` - implemented on Unix for all I/O types (files, sockets, etc)
* `IntoRawHandle` - implemented on Windows for files, processes, etc
* `IntoRawSocket` - implemented on Windows for networking types
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1174-into-raw-fd-socket-handle-traits.mdCloses#27062
* Slate these features to be stable in 1.2 instead of 1.1 (not being backported)
* Have the `FromRawFd` implementations follow the contract of the `FromRawFd`
trait by taking ownership of the primitive specified.
* Refactor the implementations slightly to remove the `unreachable!` blocks as
well as separating the stdio representation of `std::process` from
`std::sys::process`.
This commit implements a number of standard traits for the standard library's
process I/O handles. The `FromRaw{Fd,Handle}` traits are now implemented for the
`Stdio` type and the `AsRaw{Fd,Handle}` traits are now implemented for the
`Child{Stdout,Stdin,Stderr}` types.
The stability markers for these implementations mention that they are stable for
1.1 as I will nominate this commit for cherry-picking to beta.
