rust/library/std/src
bors 7fcf850d79 Auto merge of #103137 - dtolnay:readdir, r=Mark-Simulacrum
Eliminate 280-byte memset from ReadDir iterator

This guy:

1536ab1b38/library/std/src/sys/unix/fs.rs (L589)

It turns out `libc::dirent64` is quite big—https://docs.rs/libc/0.2.135/libc/struct.dirent64.html. In #103135 this memset accounted for 0.9% of the runtime of iterating a big directory.

Almost none of the big zeroed value is ever used. We memcpy a tiny prefix (19 bytes) into it, and then read just 9 bytes (`d_ino` and `d_type`) back out. We can read exactly those 9 bytes we need directly from the original entry_ptr instead.

## History

This code got added in #93459 and tweaked in #94272 and #94750.

Prior to #93459, there was no memset but a full 280 bytes were being copied from the entry_ptr.

<table><tr><td>copy 280 bytes</td></tr></table>

This was not legal because not all of those bytes might be initialized, or even allocated, depending on the length of the directory entry's name, leading to a segfault. That PR fixed the segfault by creating a new zeroed dirent64 and copying just the guaranteed initialized prefix into it.

<table><tr><td>memset 280 bytes</td><td>copy 19 bytes</td></tr></table>

However this was still buggy because it used `addr_of!((*entry_ptr).d_name)`, which is considered UB by Miri in the case that the full extent of entry_ptr is not in bounds of the same allocation. (Arguably this shouldn't be a requirement, but here we are.)

The UB got fixed by #94272 by replacing `addr_of` with some pointer manipulation based on `offset_from`, but still fundamentally the same operation.

<table><tr><td>memset 280 bytes</td><td>copy 19 bytes</td></tr></table>

Then #94750 noticed that only 9 of those 19 bytes were even being used, so we could pick out only those 9 to put in the ReadDir value.

<table><tr><td>memset 280 bytes</td><td>copy 19 bytes</td><td>copy 9 bytes</td></tr></table>

After my PR we just grab the 9 needed bytes directly from entry_ptr.

<table><tr><td>copy 9 bytes</td></tr></table>

The resulting code is more complex but I believe still worthwhile to land for the following reason. This is an extremely straightforward thing to accomplish in C and clearly libc assumes that; literally just `entry_ptr->d_name`. The extra work in comparison to accomplish it in Rust is not an example of any actual safety being provided by Rust. I believe it's useful to have uncovered that and think about what could be done in the standard library or language to support this obvious operation better.

## References

- https://man7.org/linux/man-pages/man3/readdir.3.html
2022-10-23 18:55:40 +00:00
..
backtrace Use implicit capture syntax in format_args 2022-03-10 10:23:40 -05:00
collections Make Hash{Set,Map}::with_hasher unstably const 2022-10-02 13:07:13 +01:00
env std: move "mod tests/benches" to separate files 2020-08-31 02:56:59 +00:00
error remove fn backtrace 2022-08-01 20:10:40 +00:00
f32 Semicolon after macro_rules definition. 2022-08-15 12:33:00 +02:00
f64 Semicolon after macro_rules definition. 2022-08-15 12:33:00 +02:00
ffi Guarantee try_reserve preserves the contents on error 2022-08-10 01:51:38 +09:00
fs Ignore hiberfil_sys test in CI 2022-07-18 15:06:07 +01:00
io Auto merge of #98033 - joshtriplett:is-terminal-fd-handle, r=thomcc 2022-10-15 01:42:28 +00:00
net Simplify clippy fix. 2022-09-12 19:46:51 +02:00
num rustc_expand: Mark inner #![test] attributes as soft-unstable 2020-11-20 19:35:03 +03:00
os Make the whole std::os::wasi::io module stable. 2022-10-20 14:31:11 -07:00
panic review: fix nits and move panic safety tests to the correct place 2020-09-25 23:10:24 +02:00
path make many std tests work in Miri 2022-08-18 18:07:39 -04:00
personality Interpret EH actions properly 2022-10-05 03:09:43 +01:00
prelude Finish bumping stage0 2022-05-27 07:36:17 -04:00
process Add test for issue #95178 2022-03-23 05:33:44 +00:00
sync Rollup merge of #102277 - mgeisler:rwlock, r=m-ou-se 2022-10-11 18:37:52 +09:00
sys Auto merge of #103137 - dtolnay:readdir, r=Mark-Simulacrum 2022-10-23 18:55:40 +00:00
sys_common std: use sync::Mutex for internal statics 2022-10-13 12:55:14 +02:00
thread Auto merge of #100579 - joboet:sync_mutex_everywhere, r=thomcc 2022-10-15 22:49:30 +00:00
time make many std tests work in Miri 2022-08-18 18:07:39 -04:00
alloc.rs Forbid mixing System with sytem allocator calls 2022-09-03 16:47:12 -05:00
ascii.rs Remove use of #[rustc_deprecated] 2022-04-14 01:33:13 -04:00
backtrace.rs std: use sync::Mutex for internal statics 2022-10-13 12:55:14 +02:00
env.rs env::temp_dir: fix a typo 2022-09-28 21:51:09 +02:00
error.rs remove cfg(bootstrap) 2022-09-26 10:14:45 +02:00
f32.rs Improve primitive/std docs separation and headers 2022-08-20 16:50:29 -05:00
f64.rs Improve primitive/std docs separation and headers 2022-08-20 16:50:29 -05:00
fs.rs Do not alias for fs 2022-10-10 17:05:59 -07:00
keyword_docs.rs review feedback 2022-10-07 15:21:47 +02:00
lib.rs Add IsTerminal trait to determine if a descriptor or handle is a terminal 2022-10-15 00:35:38 +01:00
macros.rs stdio: Document no support for writing to non-blocking stdio/stderr 2022-09-07 14:22:57 +01:00
num.rs Add Saturating type (based on Wrapping type) 2021-08-10 19:27:01 +02:00
panic.rs Adding backtrace off option for fuchsia targets 2022-09-14 23:54:40 +00:00
panicking.rs Fix whitespace 2022-10-06 09:55:32 +02:00
path.rs Add basename and dirname aliases 2022-10-09 21:44:44 -07:00
personality.rs Move personality functions to std 2022-08-23 16:12:58 +08:00
primitive_docs.rs array docs - advertise how to get array from slice 2022-09-10 19:37:07 -07:00
process.rs Make diagnostic for unsatisfied Termination bounds more precise 2022-10-17 12:08:46 +02:00
rt.rs Change process spawning to inherit the parent's signal mask by default 2022-10-20 14:53:38 -07:00
time.rs Fixed docs typo in library/std/src/time.rs 2022-10-19 21:49:29 -07:00