rust/library
Matthias Krüger f586a79384
Rollup merge of #121938 - blyxxyz:quadratic-vectored-write, r=Amanieu
Fix quadratic behavior of repeated vectored writes

Some implementations of `Write::write_vectored` in the standard library (`BufWriter`, `LineWriter`, `Stdout`, `Stderr`) check all buffers to calculate the total length. This is O(n) over the number of buffers.

It's common that only a limited number of buffers is written at a time (e.g. 1024 for `writev(2)`). `write_vectored_all` will then call `write_vectored` repeatedly, leading to a runtime of O(n²) over the number of buffers.

This fix is to only calculate as much as needed if it's needed.

Here's a test program:
```rust
#![feature(write_all_vectored)]

use std::fs::File;
use std::io::{BufWriter, IoSlice, Write};
use std::time::Instant;

fn main() {
    let buf = vec![b'\0'; 100_000_000];
    let mut slices: Vec<IoSlice<'_>> = buf.chunks(100).map(IoSlice::new).collect();
    let mut writer = BufWriter::new(File::create("/dev/null").unwrap());

    let start = Instant::now();
    write_smart(&slices, &mut writer);
    println!("write_smart(): {:?}", start.elapsed());

    let start = Instant::now();
    writer.write_all_vectored(&mut slices).unwrap();
    println!("write_all_vectored(): {:?}", start.elapsed());
}

fn write_smart(mut slices: &[IoSlice<'_>], writer: &mut impl Write) {
    while !slices.is_empty() {
        // Only try to write as many slices as can be written
        let res = writer
            .write_vectored(slices.get(..1024).unwrap_or(slices))
            .unwrap();
        slices = &slices[(res / 100)..];
    }
}
```
Before this change:
```
write_smart(): 6.666952ms
write_all_vectored(): 498.437092ms
```
After this change:
```
write_smart(): 6.377158ms
write_all_vectored(): 6.923412ms
```

`LineWriter` (and by extension `Stdout`) isn't fully repaired by this because it looks for newlines. I could open an issue for that after this is merged, I think it's fixable but not trivially.
2024-03-08 08:19:19 +01:00
..
alloc Rust is a proper name: rust → Rust 2024-03-07 07:49:22 +01:00
backtrace@6145fe6bac Update backtrace submodule 2023-11-21 16:33:42 +01:00
core Rollup merge of #120608 - kornelski:slice-ptr-doc, r=cuviper 2024-03-08 08:19:18 +01:00
panic_abort Cleanup windows abort_internal 2024-03-02 18:22:15 +00:00
panic_unwind miri: rename miri_start_panic → miri_start_unwind 2024-02-26 11:10:18 +01:00
portable-simd Merge commit '649110751e' into sync-portable-simd-2024-02-18 2024-02-18 10:14:03 -05:00
proc_macro Rollup merge of #120976 - matthiaskrgr:constify_TL_statics, r=lcnr 2024-03-04 22:16:30 +01:00
profiler_builtins Add profiling support to AIX 2024-02-28 17:41:12 +08:00
rtstartup library: Fix warnings in rtstartup 2024-01-06 01:32:03 +03:00
rustc-std-workspace-alloc Replace libstd, libcore, liballoc in line comments. 2022-12-30 14:00:42 +01:00
rustc-std-workspace-core
rustc-std-workspace-std
std Rollup merge of #121938 - blyxxyz:quadratic-vectored-write, r=Amanieu 2024-03-08 08:19:19 +01:00
stdarch@56087ea170 Rename wasm32-wasi-preview2 to wasm32-wasip2 2024-02-27 10:14:45 -05:00
sysroot Expose compiler-builtins-weak-intrinsics feature for -Zbuild-std 2023-06-23 11:15:34 +01:00
test Rust is a proper name: rust → Rust 2024-03-07 07:49:22 +01:00
unwind Rollup merge of #119590 - ChrisDenton:cfg-target-abi, r=Nilstrieb 2024-02-25 17:05:19 +01:00