Add #[must_use] to from_value conversions
I added two methods to the list myself. Clippy did not flag them because they take `mut` args, but neither modifies their argument.
```rust
core::str const unsafe fn from_utf8_unchecked_mut(v: &mut [u8]) -> &mut str;
std::ffi::CString unsafe fn from_raw(ptr: *mut c_char) -> CString;
```
I put a custom note on `from_raw`:
```rust
#[must_use = "call `drop(from_raw(ptr))` if you intend to drop the `CString`"]
pub unsafe fn from_raw(ptr: *mut c_char) -> CString {
```
Parent issue: #89692
r? ``@joshtriplett``
Optimize File::read_to_end and read_to_string
Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity.
`std::fs::read` and `std::fs::read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the documentation for both the `Read` trait and the `Read::read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as `impl Read` or `dyn Read` will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked `unsafe`.
- `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added `@camelid's` suggested wording from https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502.
r? `@joshtriplett`
Reading a file into an empty vector or string buffer can incur
unnecessary `read` syscalls and memory re-allocations as the buffer
"warms up" and grows to its final size. This is perhaps a necessary evil
with generic readers, but files can be read in smarter by checking the
file size and reserving that much capacity.
`std::fs::read` and `read_to_string` already perform this optimization:
they open the file, reads its metadata, and call `with_capacity` with
the file size. This ensures that the buffer does not need to be resized
and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls
`file.read_to_end` or `file.read_to_string` they do not get this
optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of
examples of this. They're found all over in unit tests, which isn't a
big deal, but there are also several real instances in the compiler and
in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the `Read` trait and the `read_to_end` method both show
this exact pattern as examples of how to use readers. What this says to
me is that this shouldn't be solved by simply fixing the instances of it
in this codebase. If it's here it's certain to be prevalent in the wider
Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and
`read_to_string` directly on `File`. This way it's no longer a minor
footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as
a bare `Read` constraint or via a `dyn Read` trait object will benefit.
For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to
`self.inner`'s methods. That way it can call `File`'s optimized
implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked
`unsafe`.
- `File::read_to_string` being more efficient means that the performance
note for `io::read_to_string` can be softened. I've added @camelid's
suggested wording from:
https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
Fix read_to_end to not grow an exact size buffer
If you know how much data to expect and use `Vec::with_capacity` to pre-allocate a buffer of that capacity, `Read::read_to_end` will still double its capacity. It needs some space to perform a read, even though that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading into a small "probe" buffer instead. If that read returns `0` then it's confirmed that the buffer was the perfect size. If it doesn't, the probe buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in `initial_buffer_size`.
Curiously, there was a unit test that specifically checked that `Read::read_to_end` *does* over-allocate. I removed that test, too.
Fix spacing of links in inline code.
Similar to #80733, but the focus is different. This PR eliminates all occurrences of pieced-together inline code blocks like [`Box`]`<`[`Option`]`<T>>` and replaces them with good-looking ones (using HTML-syntax), like <code>[Box]<[Option]\<T>></code>. As far as I can tell, I should’ve found all of these in the standard library (regex search with `` r"`\]`|`\[`" ``) \[except for in `core::convert` where I’ve noticed other things in the docs that I want to fix in a separate PR]. In particular, unlike #80733, I’ve added almost no new instance of inline code that’s broken up into multiple links (or some link and some link-free part). I also added tooltips (the stuff in quotes for the markdown link listings) in places that caught my eye, but that’s by no means systematic, just opportunistic.
[Box]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[`Box`]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[Option]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
[`Option`]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
Context: I got annoyed by repeatedly running into new misformatted inline code while reading the standard library docs. I know that once issue #83997 (and/or related ones) are resolved, these changes become somewhat obsolete, but I fail to notice much progress on that end right now.
r? `@jyn514`
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync}
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::string
----------
Fix spacing for links inside code blocks in alloc::vec
----------
Fix spacing for links inside code blocks in core::option
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in core::result
----------
Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll}
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path}
----------
Fix spacing for links inside code blocks in std::{collections, time}
----------
Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str}
----------
Fix spacing for links inside code blocks, and improve link tooltips in std::ffi
----------
Fix spacing for links inside code blocks, and improve a few link tooltips
in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}}
----------
Fix typo in link to `into` for `OsString` docs
----------
Remove tooltips that will probably become redundant in the future
----------
Apply suggestions from code review
Replacing `…std/primitive.reference.html` paths with just `reference`
Co-authored-by: Joshua Nelson <github@jyn.dev>
----------
Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
If you know how much data to expect and use `Vec::with_capacity` to
pre-allocate a buffer of that capacity, `Read::read_to_end` will still
double its capacity. It needs some space to perform a read, even though
that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into
memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading
into a small "probe" buffer instead. If that read returns `0` then it's
confirmed that the buffer was the perfect size. If it doesn't, the probe
buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be
removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the
previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in
`initial_buffer_size`.
Curiously, there was a unit test that specifically checked that
`Read::read_to_end` *does* over-allocate. I removed that test, too.
Stabilise BufWriter::into_parts
The FCP for this has already completed, in #80690.
This was just blocked on #85901 (which changed the name), which is now merged. The original stabilisation MR was #84770 but that has a lot of noise in it, and I also accidentally deleted the branch while trying to tidy up. So here is a new MR. Sorry for the noise.
Closes#80690
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Replace read_to_string with read_line in Stdin example
The current example results in infinitely reading from stdin, which can confuse newcomers trying to read from stdin.
(`@razmag` encountered this while learning the language from the docs)
impl Default, Copy, Clone for std::io::Sink and Empty
The omission of `Sink: Default` is causing me a slight inconvenience in a test harness. There seems little reason for this and `Empty` not to be `Clone` and `Copy` too.
I have made all three of these insta-stable, because:
AIUI `Copy` can only be derived, and I was not able to find any examples of how to unstably derive it. I think it is probably not possible.
I hunted through the git history for precedent and found
> 79b8ad84c8
> Implement `Copy` for `IoSlice`
> https://github.com/rust-lang/rust/pull/69403
which was also insta-stable.
Fix may not to appropriate might not or must not
I went through and changed occurrences of `may not` to be more explicit with `might not` and `must not`.
I looked in stdlib and as @BurntSushi thought, `raw` is generally
used for raw pointers, or other hazardous kinds of thing. stdlib does
not have `into_parts` apart from the one I added to `IntoInnerError`.
I did an ad-hoc search of the rustdocs for my current game project
Otter, which includes quite a large number of dependencies.
`into_parts` seems heavily used for things quite like this.
So change this name.
Suggested-by: Andrew Gallant <jamslam@gmail.com>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
I didn't notice the submodule, which means I failed to re-export this
to make it actually-public.
Reported-by: Andrew Gallant <jamslam@gmail.com>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Stabilize `into_parts()` and `into_error()`
This stabilizes `IntoInnerError`'s `into_parts()` and `into_error()` methods, currently gated behind the `io_into_inner_error_parts` feature. The FCP has [already completed.](https://github.com/rust-lang/rust/issues/79704#issuecomment-880652967)
Closes#79704.
add `Stdin::lines`, `Stdin::split` forwarder methods
Add forwarder methods `Stdin::lines` and `Stdin::split`, which consume
and lock a `Stdin` handle, and forward on to the corresponding `BufRead`
methods. This should make it easier for beginners to use those iterator
constructors without explicitly dealing with locks or lifetimes.
Replaces #86412.
~~Based on #86846 to get the tracking issue number for the `stdio_locked` feature.~~ Rebased after merge, so it's only one commit now.
r? `@joshtriplett`
`@rustbot` label +A-io +C-enhancement +D-newcomer-roadblock +T-libs-api
Add diagnostic items for Clippy
This adds a bunch of diagnostic items to `std`/`core`/`alloc` functions, structs and traits used in Clippy. The actual refactorings in Clippy to use these items will be done in a different PR in Clippy after the next sync.
This PR doesn't include all paths Clippy uses, I've only gone through the first 85 lines of Clippy's [`paths.rs`](ecf85f4bdc/clippy_utils/src/paths.rs) (after rust-lang/rust-clippy#7466) to get some feedback early on. I've also decided against adding diagnostic items to methods, as it would be nicer and more scalable to access them in a nicer fashion, like adding a `is_diagnostic_assoc_item(did, sym::Iterator, sym::map)` function or something similar (Suggested by `@camsteffen` [on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/147480-t-compiler.2Fwg-diagnostics/topic/Diagnostic.20Item.20Naming.20Convention.3F/near/225024603))
There seems to be some different naming conventions when it comes to diagnostic items, some use UpperCamelCase (`BinaryHeap`) and some snake_case (`hashmap_type`). This PR uses UpperCamelCase for structs and traits and snake_case with the module name as a prefix for functions. Any feedback on is this welcome.
cc: rust-lang/rust-clippy#5393
r? `@Manishearth`
Add forwarder methods `Stdin::lines` and `Stdin::split`, which consume
and lock a `Stdin` handle, and forward on to the corresponding `BufRead`
methods. This should make it easier for beginners to use those iterator
constructors without explicitly dealing with locks or lifetimes.
stdio_locked: add tracking issue
Add the tracking issue number #86845 to the stability attributes for the implementation in #86799.
r? `@joshtriplett`
`@rustbot` label +A-io +C-cleanup +T-libs-api
Remove unstable `io::Cursor::remaining`
Adding `io::Cursor::remaining` in #86037 caused a conflict with the implementation of `bytes::Buf` for `io::Cursor`, leading to an error in nightly, see https://github.com/rust-lang/rust/issues/86369#issuecomment-867723485.
This fixes the error by temporarily removing the `remaining` function.
r? `@yaahc`
Stabilize `Seek::rewind()`
This stabilizes `Seek::rewind`. It seemed to fit into one of the existing tests, so I extended that test rather than adding a new one.
Closes#85149.
add owned locked stdio handles
Add stderr_locked, stdin_locked, and stdout_locked free functions
to obtain owned locked stdio handles in a single step. Also add
into_lock methods to consume a stdio handle and return an owned
lock. These methods will make it easier to use locked stdio
handles without having to deal with lifetime problems or keeping
bindings to the unlocked handles around.
Fixes#85383; enables #86412.
r? `@joshtriplett`
`@rustbot` label +A-io +C-enhancement +D-newcomer-roadblock +T-libs-api
More ErrorKinds for common errnos
From the commit message of the main commit here (as revised):
```
There are a number of IO error situations which it would be very
useful for Rust code to be able to recognise without having to resort
to OS-specific code. Taking some Unix examples, `ENOTEMPTY` and
`EXDEV` have obvious recovery strategies. Recently I was surprised to
discover that `ENOSPC` came out as `ErrorKind::Other`.
Since I am familiar with Unix I reviwed the list of errno values in
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/errno.h.html
Here, I add those that most clearly seem to be needed.
`@CraftSpider` provided information about Windows, and references, which
I have tried to take into account.
This has to be insta-stable because we can't sensibly have a different
set of ErrorKinds depending on a std feature flag.
I have *not* added these to the mapping tables for any operating
systems other than Unix and Windows. I hope that it is OK to add them
now for Unix and Windows now, and maybe add them to other OS's mapping
tables as and when someone on that OS is able to consider the
situation.
I adopted the general principle that it was usually a bad idea to map
two distinct error values to the same Rust error code. I notice that
this principle is already violated in the case of `EACCES` and
`EPERM`, which both map to `PermissionDenied`. I think this was
probably a mistake but it would be quite hard to change now, so I
don't propose to do anything about that.
However, for Windows, there are sometimes different error codes for
identical situations. Eg there are WSA* versions of some error
codes as well as ERROR_* ones. Also Windows seems to have a great
many more erorr codes. I don't know precisely what best practice
would be for Windows.
```
<strike>
```
Errno values I wasn't sure about so *haven't* included:
EMFILE ENFILE ENOBUFS ENOLCK:
These are all fairly Unix-specific resource exhaustion situations.
In practice it seemed not very likely to me that anyone would want
to handle these differently to `Other`.
ENOMEM ERANGE EDOM EOVERFLOW
Normally these don't get exposed to the Rust callers I hope. They
don't tend to come out of filesystem APIs.
EILSEQ
Hopefully Rust libraries open files in binary mode and do the
converstion in Rust. So Rust code ought not to be exposed to
EILSEQ.
EIO
The range of things that could cause this is troublesome. I found
it difficult to describe. I do think it would be useful to add this
at some point, because EIO on a filesystem operation is much more
serious than most other errors.
ENETDOWN
I wasn't sure if this was useful or, indeed, if any modern systems
use it.
ENOEXEC
It is not clear to me how a Rust program could respond to this. It
seems rather niche.
EPROTO ENETRESET ENODATA ENOMSG ENOPROTOOPT ENOSR ENOSTR ETIME
ENOTRECOVERABLE EOWNERDEAD EBADMSG EPROTONOSUPPORT EPROTOTYPE EIDRM
These are network or STREAMS related errors which I have never in
my own Unix programming found the need to do anything with. I think
someone who understands these better should be the one to try to
find good Rust names and descriptions for them.
ENOTTY ENXIO ENODEV EOPNOTSUPP ESRCH EALREADY ECANCELED ECHILD
EINPROGRESS
These are very hard to get unless you're already doing something
very Unix-specific, in which case the raw_os_error interface is
probably more suitable than relying on the Rust ErrorKind mapping.
EFAULT EBADF
These would seem to be the result of application UB.
```
</strike>
<i>(omitted errnos are discussed below, especially in https://github.com/rust-lang/rust/pull/79965#issuecomment-810468334)
