rust/library/std/src
bors 30e49a9ead Auto merge of #75272 - the8472:spec-copy, r=KodrAus
specialize io::copy to use copy_file_range, splice or sendfile

Fixes #74426.
Also covers #60689 but only as an optimization instead of an official API.

The specialization only covers std-owned structs so it should avoid the problems with #71091

Currently linux-only but it should be generalizable to other unix systems that have sendfile/sosplice and similar.

There is a bit of optimization potential around the syscall count. Right now it may end up doing more syscalls than the naive copy loop when doing short (<8KiB) copies between file descriptors.

The test case executes the following:

```
[pid 103776] statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=17, ...}) = 0
[pid 103776] write(4, "wxyz", 4)        = 4
[pid 103776] write(4, "iklmn", 5)       = 5
[pid 103776] copy_file_range(3, NULL, 4, NULL, 5, 0) = 5

```

0-1 `stat` calls to identify the source file type. 0 if the type can be inferred from the struct from which the FD was extracted
𝖬 `write` to drain the `BufReader`/`BufWriter` wrappers. only happen when buffers are present. 𝖬 ≾ number of wrappers present. If there is a write buffer it may absorb the read buffer contents first so only result in a single write. Vectored writes would also be an option but that would require more invasive changes to `BufWriter`.
𝖭 `copy_file_range`/`splice`/`sendfile` until file size, EOF or the byte limit from `Take` is reached. This should generally be *much* more efficient than the read-write loop and also have other benefits such as DMA offload or extent sharing.

## Benchmarks

```

OLD

test io::tests::bench_file_to_file_copy         ... bench:      21,002 ns/iter (+/- 750) = 6240 MB/s    [ext4]
test io::tests::bench_file_to_file_copy         ... bench:      35,704 ns/iter (+/- 1,108) = 3671 MB/s  [btrfs]
test io::tests::bench_file_to_socket_copy       ... bench:      57,002 ns/iter (+/- 4,205) = 2299 MB/s
test io::tests::bench_socket_pipe_socket_copy   ... bench:     142,640 ns/iter (+/- 77,851) = 918 MB/s

NEW

test io::tests::bench_file_to_file_copy         ... bench:      14,745 ns/iter (+/- 519) = 8889 MB/s    [ext4]
test io::tests::bench_file_to_file_copy         ... bench:       6,128 ns/iter (+/- 227) = 21389 MB/s   [btrfs]
test io::tests::bench_file_to_socket_copy       ... bench:      13,767 ns/iter (+/- 3,767) = 9520 MB/s
test io::tests::bench_socket_pipe_socket_copy   ... bench:      26,471 ns/iter (+/- 6,412) = 4951 MB/s
```
2020-11-14 12:01:55 +00:00
..
backtrace std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
collections Rollup merge of #77996 - tkaitchuck:master, r=m-ou-se 2020-11-13 15:26:10 +01:00
env std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
error std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
f32 std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
f64 std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
ffi Fix bootstrap doctest failure 2020-10-26 22:09:47 -04:00
fs Rollup merge of #78026 - sunfishcode:symlink-hard-link, r=dtolnay 2020-11-09 01:13:28 +01:00
io Auto merge of #75272 - the8472:spec-copy, r=KodrAus 2020-11-14 12:01:55 +00:00
lazy std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
memchr std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
net Rollup merge of #78208 - liketechnik:issue-69399, r=oli-obk 2020-10-25 18:43:40 +09:00
num std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
os Take sys/vxworks/{fd,fs,io} from sys/unix instead. 2020-10-16 06:19:00 +02:00
panic review: fix nits and move panic safety tests to the correct place 2020-09-25 23:10:24 +02:00
path std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
prelude Fix incorrect link in prelude 2020-09-02 17:38:21 -04:00
process std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
sync Convert a bunch of intra-doc links 2020-11-07 12:50:57 -08:00
sys Auto merge of #75272 - the8472:spec-copy, r=KodrAus 2020-11-14 12:01:55 +00:00
sys_common Rollup merge of #77648 - fusion-engineering-forks:static-mutex, r=dtolnay 2020-10-16 02:10:15 +02:00
thread Convert a bunch of intra-doc links 2020-11-07 12:50:57 -08:00
time Duration::zero() -> Duration::ZERO 2020-10-21 20:44:03 -07:00
alloc.rs Add missing newline to error message of the default OOM hook 2020-11-10 00:15:07 +08:00
ascii.rs Convert many files to intra-doc links 2020-09-02 17:37:40 -04:00
backtrace.rs For backtrace, use StaticMutex instead of a raw sys Mutex. 2020-10-07 13:59:03 +02:00
env.rs Improve docs for std::env::args() 2020-09-04 14:00:09 -07:00
error.rs Rename AllocErr to AllocError 2020-09-28 14:51:03 -04:00
f32.rs Rollup merge of #77099 - tspiteri:exp_m1-examples, r=m-ou-se 2020-10-31 09:49:32 +01:00
f64.rs Rollup merge of #77099 - tspiteri:exp_m1-examples, r=m-ou-se 2020-10-31 09:49:32 +01:00
fs.rs Auto merge of #75272 - the8472:spec-copy, r=KodrAus 2020-11-14 12:01:55 +00:00
future.rs Fix stabilization marker for future_readiness_fns 2020-09-15 23:12:08 +02:00
keyword_docs.rs Clean up docs for 'as' keyword 2020-11-04 16:05:55 -08:00
lazy.rs Simplify SyncOnceCell's take and drop. 2020-09-12 14:00:38 +02:00
lib.rs Auto merge of #75272 - the8472:spec-copy, r=KodrAus 2020-11-14 12:01:55 +00:00
macros.rs Use ? in core/std macros 2020-10-26 07:15:37 +09:00
memchr.rs std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
num.rs std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
panic.rs Add tracking issue number for panic_any. 2020-10-28 21:23:45 +01:00
panicking.rs Auto merge of #78227 - SergioBenitez:test-stdout-threading, r=m-ou-se 2020-10-27 11:43:18 +00:00
path.rs Clean up intra-doc links in std::path 2020-10-27 20:54:30 -07:00
primitive_docs.rs Rollup merge of #78757 - camelid:crate-link-text, r=jyn514 2020-11-05 10:30:02 +01:00
process.rs Apply suggestions from code review 2020-10-31 17:28:44 +01:00
rt.rs Prevent __rust_begin_short_backtrace frames from being tail-call optimised away 2020-08-07 19:31:25 +01:00
time.rs Changed unwrap_or to unwrap_or_else in some places. 2020-11-10 20:07:47 -08:00