Optimize `read_to_end`.
This patch makes `read_to_end` use Vec's memory-growth pattern rather
than using a custom pattern.
This has some interesting effects:
- If memory is reserved up front, `read_to_end` can be faster, as it
starts reading at the buffer size, rather than always starting at 32
bytes. This speeds up file reading by 2x in one of my use cases.
- It can reduce the number of syscalls when reading large files.
Previously, `read_to_end` would settle into a sequence of 8192-byte
reads. With this patch, the read size follows Vec's allocation
pattern. For example, on a 16MiB file, it can do 21 read syscalls
instead of 2057. In simple benchmarks of large files though, overall
speed is still dominated by the actual I/O.
- A downside is that Read implementations that don't implement
`initializer()` may see increased memory zeroing overhead.
I benchmarked this on a variety of data sizes, with and without
preallocated buffers. Most benchmarks see no difference, but reading
a small/medium file with a pre-allocated buffer is faster.
show in docs whether the return type of a function impls Iterator/Read/Write
Closes#25928
This PR makes it so that when rustdoc documents a function, it checks the return type to see whether it implements a handful of specific traits. If so, it will print the impl and any associated types. Rather than doing this via a whitelist within rustdoc, i chose to do this by a new `#[doc]` attribute parameter, so things like `Future` could tap into this if desired.
### Known shortcomings
~~The printing of impls currently uses the `where` class over the whole thing to shrink the font size relative to the function definition itself. Naturally, when the impl has a where clause of its own, it gets shrunken even further:~~ (This is no longer a problem because the design changed and rendered this concern moot.)
The lookup currently just looks at the top-level type, not looking inside things like Result or Option, which renders the spotlights on Read/Write a little less useful:
<details><summary>`File::{open, create}` don't have spotlight info (pic of old design)</summary>
</details>
All three of the initially spotlighted traits are generically implemented on `&mut` references. Rustdoc currently treats a `&mut T` reference-to-a-generic as an impl on the reference primitive itself. `&mut Self` counts as a generic in the eyes of rustdoc. All this combines to create this lovely scene on `Iterator::by_ref`:
<details><summary>`Iterator::by_ref` spotlights Iterator, Read, and Write (pic of old design)</summary>
</details>
add doc for doing `Read` from `&str`
This information can be found on [stackoverflow](https://stackoverflow.com/questions/32674905/pass-string-to-function-taking-read-trait) but I think it would be beneficial if it was documented in the `Read` trait itself.
I had an *extremely* hard time finding this information, and "mocking" a reader with a string is an EXTREMELY common thing (I believe).
impl From for Mutex and RwLock
I felt that these implementations were missing, because doing `x.into()` works for other smart containers (such as `RefCell`), and in general I would say that the conversion makes sense.
This commit adds a new target to the compiler: wasm32-unknown-unknown. This
target is a reimagining of what it looks like to generate WebAssembly code from
Rust. Instead of using Emscripten which can bring with it a weighty runtime this
instead is a target which uses only the LLVM backend for WebAssembly and a
"custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than
the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker
is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this
target.
* Most of the standard library is stubbed out to return an error, but anything
related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new
target.
This target is currently somewhat janky due to how linking works. The "linking"
is currently unconditional whole program LTO (aka LLVM is being used as a
linker). Naturally that means compiling programs is pretty slow! Eventually
though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can
act as a catalyst for further experimentation in Rust with WebAssembly. Breaking
changes are very likely to land to this target, so it's not recommended to rely
on it in any critical capacity yet. We'll let you know when it's "production
ready".
---
Currently testing-wise this target is looking pretty good but isn't complete.
I've got almost the entire `run-pass` test suite working with this target (lots
of tests ignored, but many passing as well). The `core` test suite is still
getting LLVM bugs fixed to get that working and will take some time. Relatively
simple programs all seem to work though!
---
It's worth nothing that you may not immediately see the "smallest possible wasm
module" for the input you feed to rustc. For various reasons it's very difficult
to get rid of the final "bloat" in vanilla rustc (again, a real linker should
fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various
integration points if you've got better ideas of how to approach them!
This has been discussed in #39658. It's a bit ambiguous how those
methods work for a sequence of ascii values. We prefer users writing
`s.iter().all(|b| b.is_ascii_...())` explicitly.
The AsciiExt methods still exist and are implemented for `str`
and `[u8]`. We will deprecated or remove those later.
This patch makes `read_to_end` use Vec's memory-growth pattern rather
than using a custom pattern.
This has some interesting effects:
- If memory is reserved up front, `read_to_end` can be faster, as it
starts reading at the buffer size, rather than always starting at 32
bytes. This speeds up file reading by 2x in one of my use cases.
- It can reduce the number of syscalls when reading large files.
Previously, `read_to_end` would settle into a sequence of 8192-byte
reads. With this patch, the read size follows Vec's allocation
pattern. For example, on a 16MiB file, it can do 21 read syscalls
instead of 2057. In simple benchmarks of large files though, overall
speed is still dominated by the actual I/O.
- A downside is that Read implementations that don't implement
`initializer()` may see increased memory zeroing overhead.
I benchmarked this on a variety of data sizes, with and without
preallocated buffers. Most benchmarks see no difference, but reading
a small/medium file with a pre-allocated buffer is faster.
Use getrandom syscall for all Linux and Android targets.
I suppose we can use it in all Linux and Android targets. In function `is_getrandom_available` is checked if the syscall is available (getrandom syscall was add in version 3.17 of Linux kernel), if the syscall is not available `fill_bytes` fallback to reading from `/dev/urandom`.
Update libc to include getrandom related constants.
Essentially renames recv_max_until to recv_deadline (mostly copying recv_timeout
documentation). This function is useful to avoid the often unnecessary call to
Instant::now in recv_timeout (e.g. when the user already has a deadline). A
concrete example would be something along those lines:
```rust
use std::sync::mpsc::Receiver;
use std::time::{Duration, Instant};
/// Reads a batch of elements
///
/// Returns as soon as `max_size` elements have been received or `timeout` expires.
fn recv_batch_timeout<T>(receiver: &Receiver<T>, timeout: Duration, max_size: usize) -> Vec<T> {
recv_batch_deadline(receiver, Instant::now() + timeout, max_size)
}
/// Reads a batch of elements
///
/// Returns as soon as `max_size` elements have been received or `deadline` is reached.
fn recv_batch_deadline<T>(receiver: &Receiver<T>, deadline: Instant, max_size: usize) -> Vec<T> {
let mut result = Vec::new();
while let Ok(x) = receiver.recv_deadline(deadline) {
result.push(x);
if result.len() == max_size {
break;
}
}
result
}
```
Redox: Use futex timeout to implement CondVar::wait_timeout
`CondVar::wait_timeout` is implemented by supplying a `TimeSpec` pointer to `futex`. In addition, all calls to `unimplemented!()` have been removed from the Redox `sys` module.
Related to https://github.com/rust-lang/rust/pull/45892
Redox: Return true from Path::is_absolute if a Path contains root or a scheme
In Redox, different subsystems have different filesystem paths. However, the majority of applications using the `Path::is_absolute` function really only want to know if a path is absolute from the perspective of the scheme it is currently running in, usually `file:`. This makes both `file:/` and `/` return `true` from `Path::is_absolute`, meaning that most code does not have to check if it is running on Redox.
Code that wants to know if a path contains a scheme can implement such a check on its own.
Related to https://github.com/rust-lang/rust/pull/45893
Addressed issues raised in #44286. (`OccupiedEntry::replace_entry`)
This commit renames the `replace` function to `replace_entry`, and
creates a seperate `replace_key` function for `OccupiedEntry`. The
original `replace` function did not solve the use-case where the
key needed to be replaced, but not the value. Documentation and
naming has also been updated to better reflect what the original
replace function does.
This commit renames the `replace` function to `replace_entry`, and
creates a seperate `replace_key` function for `OccupiedEntry`. The
original `replace` function did not solve the use-case where the
key needed to be replaced, but not the value. Documentation and
naming has also been updated to better reflect what the original
replace function does.
This commit removes usage of the `libc` crate in "portable" modules like
those at the top level and `sys_common`. Instead common types like `*mut
u8` or `u32` are used instead of `*mut c_void` or `c_int` as well as
switching to platform-specific functions like `sys::strlen` instead of
`libc::strlen`.
This commit moves the `f32::cmath` and `f64::cmath` modules into the
`sys` module. Note that these are not publicly exported modules, simply
implementation details. These modules are already platform-specific with
shims on MSVC and this is mostly just a reflection of that reality. This
should also help cut down on `#[cfg]` traffic if platforms are brought on
which don't directly support these functions.
This commit removes the reexport of `EBADF_ERR` as a constant from
libstd's portability facade, instead opting for a platform-specific
function that specifically queries an `io::Error`. Not all platforms may
have a constant for this, so it makes the intent a little more clear
that a code need not be supplied, just an answer to a query.
This commit removes the `rand` crate from the standard library facade as
well as the `__rand` module in the standard library. Neither of these
were used in any meaningful way in the standard library itself. The only
need for randomness in libstd is to initialize the thread-local keys of
a `HashMap`, and that unconditionally used `OsRng` defined in the
standard library anyway.
The cruft of the `rand` crate and the extra `rand` support in the
standard library makes libstd slightly more difficult to port to new
platforms, namely WebAssembly which doesn't have any randomness at all
(without interfacing with JS). The purpose of this commit is to clarify
and streamline randomness in libstd, focusing on how it's only required
in one location, hashmap seeds.
Note that the `rand` crate out of tree has almost always been a drop-in
replacement for the `rand` crate in-tree, so any usage (accidental or
purposeful) of the crate in-tree should switch to the `rand` crate on
crates.io. This then also has the further benefit of avoiding
duplication (mostly) between the two crates!
Warn about lack of args glob expansion in Windows shell
Because all shells on Linux/macOS expand globs, and even MinGW on Windows emulates this behavior, it's easy to forget that Windows by itself doesn't support glob expansion. This PR documents this cross-platform difference.
RwLock guards are Sync if T is
Currently, the compiler requires `T` to also be `Send`. There is no reason for
that. `&Rw{Read,Write}LockGuard` only provides a shared referenced to `T`, sending
that across threads is safe if `T` is `Sync`.
Cc @oconnor663
Implement is_empty() for BufReader
Simple implementation of `is_empty` method for BufReader so it is possible to tell whether there is any data in its buffer.
I didn't know correct stability annotation to place on the function. Presumably there is no reason to place this feature behind a feature flag, but I wasn't sure how to tag it as an unstable feature without that.
CC: #45323
