Stabilize Iterator::flatten in 1.29, fixes#48213.
This PR stabilizes [`Iterator::flatten`](https://doc.rust-lang.org/nightly/std/iter/trait.Iterator.html#method.flatten) in *version 1.29* (1.28 goes to beta in 10 days, I don't think there's enough time to land it in that time, but let's see...).
Tracking issue is: #48213.
cc @bluss re. itertools.
r? @SimonSapin
ping @pietroalbini -- let's do a crater run when this passes CI :)
Add Ref/RefMut map_split method
As proposed [here](https://internals.rust-lang.org/t/make-refcell-support-slice-splitting/7707).
TLDR: Add a `map_split` method that allows multiple `RefMut`s to exist simultaneously so long as they refer to non-overlapping regions of the original `RefCell`. This is useful for things like the slice `split_at_mut` method.
Implement [T]::align_to
Note that this PR deviates from what is accepted by RFC slightly by making `align_offset` to return an offset in elements, rather than bytes. This is necessary to sanely support `[T]::align_to` and also simply makes more sense™. The caveat is that trying to align a pointer of ZST is now an equivalent to `is_aligned` check, rather than anything else (as no number of ZST elements will align a misaligned ZST pointer).
It also implements the `align_to` slightly differently than proposed in the RFC to properly handle cases where size of T and U aren’t co-prime.
Furthermore, a promise is made that the slice containing `U`s will be as large as possible (contrary to the RFC) – otherwise the function is quite useless.
The implementation uses quite a few underhanded tricks and takes advantage of the fact that alignment is a power-of-two quite heavily to optimise the machine code down to something that results in as few known-expensive instructions as possible. Currently calling `ptr.align_offset` with an unknown-at-compile-time `align` results in code that has just a single "expensive" modulo operation; the rest is "cheap" arithmetic and bitwise ops.
cc https://github.com/rust-lang/rust/issues/44488 @oli-obk
As mentioned in the commit message for align_offset, many thanks go to Chris McDonald.
All other tests of libcore reside in the tests/ directory,
too. Apparently the tests of `time.rs` weren't run before, at
least not by `x.py test src/libcore`.
Add Cell::update
This commit adds a new method `Cell::update`, which applies a function to the value inside the cell.
Previously discussed in: https://github.com/rust-lang/rfcs/issues/2171
### Motivation
Updating `Cell`s is currently a bit verbose. Here are several real examples (taken from rustc and crossbeam):
```rust
self.print_fuel.set(self.print_fuel.get() + 1);
self.diverges.set(self.diverges.get() | Diverges::Always);
let guard_count = self.guard_count.get();
self.guard_count.set(guard_count.checked_add(1).unwrap());
if guard_count == 0 {
// ...
}
```
With the addition of the new method `Cell::update`, this code can be simplified to:
```rust
self.print_fuel.update(|x| x + 1);
self.diverges.update(|x| x | Diverges::Always);
if self.guard_count.update(|x| x.checked_add(1).unwrap()) == 1 {
// ...
}
```
### Unresolved questions
1. Should we return the old value instead of the new value (like in `fetch_add` and `fetch_update`)?
2. Should the return type simply be `()`?
3. Naming: `update` vs `modify` vs `mutate` etc.
cc @SimonSapin
Add inherent methods in libcore for [T], [u8], str, f32, and f64
# Background
Primitive types are defined by the language, they don’t have a type definition like `pub struct Foo { … }` in any crate. So they don’t “belong” to any crate as far as `impl` coherence is concerned, and on principle no crate would be able to define inherent methods for them, without a trait. Since we want these types to have inherent methods anyway, the standard library (with cooperation from the compiler) bends this rule with code like [`#[lang = "u8"] impl u8 { /*…*/ }`](https://github.com/rust-lang/rust/blob/1.25.0/src/libcore/num/mod.rs#L2244-L2245). The `#[lang]` attribute is permanently-unstable and never intended to be used outside of the standard library.
Each lang item can only be defined once. Before this PR there is one impl-coherence-rule-bending lang item per primitive type (plus one for `[u8]`, which overlaps with `[T]`). And so one `impl` block each. These blocks for `str`, `[T]` and `[u8]` are in liballoc rather than libcore because *some* of the methods (like `<[T]>::to_vec(&self) -> Vec<T> where T: Clone`) need a global memory allocator which we don’t want to make a requirement in libcore. Similarly, `impl f32` and `impl f64` are in libstd because some of the methods are based on FFI calls to C’s `libm` and we want, as much as possible, libcore not to require “runtime support”.
In libcore, the methods of `str` and `[T]` that don’t allocate are made available through two **unstable traits** `StrExt` and `SliceExt` (so the traits can’t be *named* by programs on the Stable release channel) that have **stable methods** and are re-exported in the libcore prelude (so that programs on Stable can *call* these methods anyway). Non-allocating `[u8]` methods are not available in libcore: https://github.com/rust-lang/rust/issues/45803. Some `f32` and `f64` methods are in an unstable `core::num::Float` trait with stable methods, but that one is **not in the libcore prelude**. (So as far as Stable programs are concerns it doesn’t exist, and I don’t know what the point was to mark these methods `#[stable]`.)
https://github.com/rust-lang/rust/issues/32110 is the tracking issue for these unstable traits.
# High-level proposal
Since the standard library is already bending the rules, why not bend them *a little more*? By defining a few additional lang items, the compiler can allow the standard library to have *two* `impl` blocks (in different crates) for some primitive types.
The `StrExt` and `SliceExt` traits still exist for now so that we can bootstrap from a previous-version compiler that doesn’t have these lang items yet, but they can be removed in next release cycle. (`Float` is used internally and needs to be public for libcore unit tests, but was already `#[doc(hidden)]`.) I don’t know if https://github.com/rust-lang/rust/issues/32110 should be closed by this PR, or only when the traits are entirely removed after we make a new bootstrap compiler.
# Float methods
Among the methods of the `core::num::Float` trait, three are based on LLVM intrinsics: `abs`, `signum`, and `powi`. PR https://github.com/rust-lang/rust/pull/27823 “Remove dependencies on libm functions from libcore” moved a bunch of `core::num::Float` methods back to libstd, but left these three behind. However they aren’t specifically discussed in the PR thread. The `compiler_builtins` crate defines `__powisf2` and `__powidf2` functions that look like implementations of `powi`, but I couldn’t find a connection with the `llvm.powi.f32` and `llvm.powi.f32` intrinsics by grepping through LLVM’s code.
In discussion starting at https://github.com/rust-lang/rust/issues/32110#issuecomment-370647922 Alex says that we do not want methods in libcore that require “runtime support”, but it’s not clear whether that applies to these `abs`, `signum`, or `powi`. In doubt, I’ve **removed** them for the trait and moved them to inherent methods in libstd for now. We can move them back later (or in this PR) if we decide that’s appropriate.
# Change details
For users on the Stable release channel:
* I believe this PR does not make any breaking change
* Some methods for `[u8]`, `f32`, and `f64` are newly available to `#![no_std]` users (fixes https://github.com/rust-lang/rust/issues/45803)
* There should be no visible change for `std` users in terms of what programs compile or what their behavior is. (Only in compiler error messages, possibly.)
For Nightly users, additionally:
* The unstable `StrExt` and `SliceExt` traits are gone
* Their methods are now inherent methods of `str` and `[T]` (so only code that explicitly named the traits should be affected, not "normal" method calls)
* The `abs`, `signum` and `powi` methods of the `Float` trait are gone
* The `Float` trait’s unstable feature name changed to `float_internals` with no associated tracking issue, to reflect it being a permanently unstable implementation detail rather than a public API on a path to stabilization.
* Its remaining methods are now inherent methods of `f32` and `f64`.
-----
CC @rust-lang/libs for the API changes, @rust-lang/compiler for the new lang items
Add Iterator::find_map
I'd like to propose to add `find_map` method to the `Iterator`: an occasionally useful utility, which relates to `filter_map` in the same way that `find` relates to `filter`.
`find_map` takes an `Option`-returning function, applies it to the elements of the iterator, and returns the first non-`None` result. In other words, `find_map(f) == filter_map(f).next()`.
Why do we want to add a function to the `Iterator`, which can be trivially expressed as a combination of existing ones? Observe that `find(f) == filter(f).next()`, so, by the same logic, `find` itself is unnecessary!
The more positive argument is that desugaring of `find[_map]` in terms of `filter[_map]().next()` is not super obvious, because the `filter` operation reads as if it is applies to the whole collection, although in reality we are interested only in the first element. That is, the jump from "I need a **single** result" to "let's use a function which maps **many** values to **many** values" is a non-trivial speed-bump, and causes friction when reading and writing code.
Does the need for `find_map` arise in practice? Yes!
* Anecdotally, I've more than once searched the docs for the function with `[T] -> (T -> Option<U>) -> Option<U>` signature.
* The direct cause for this PR was [this](https://github.com/rust-lang/cargo/pull/5187/files/1291c50e86ed4b31db0c76de03a47a5d0074bbd7#r174934173) discussion in Cargo, which boils down to "there's some pattern that we try to express here, but current approaches looks non-pretty" (and the pattern is `filter_map`
* There are several `filter_map().next` combos in Cargo: [[1]](545a4a2c93/src/cargo/ops/cargo_new.rs (L585)), [[2]](545a4a2c93/src/cargo/core/resolver/mod.rs (L1130)), [[3]](545a4a2c93/src/cargo/ops/cargo_rustc/mod.rs (L1086)).
* I've also needed similar functionality in `Kotlin` several times. There, it is expressed as `mapNotNull {}.firstOrNull`, as can be seen [here](ee8bdb4e07/src/main/kotlin/org/rust/cargo/project/model/impl/CargoProjectImpl.kt (L154)), [here](ee8bdb4e07/src/main/kotlin/org/rust/lang/core/resolve/ImplLookup.kt (L444)) [here](ee8bdb4e07/src/main/kotlin/org/rust/ide/inspections/RsLint.kt (L38)) and [here](ee8bdb4e07/src/main/kotlin/org/rust/cargo/toolchain/RustToolchain.kt (L74)) (and maybe in some other cases as well)
Note that it is definitely not among the most popular functions (it definitely is less popular than `find`), but, for example it (in case of Cargo) seems to be more popular than `rposition` (1 occurrence), `step_by` (zero occurrences) and `nth` (three occurrences as `nth(0)` which probably should be replaced with `next`).
Do we necessary need this function in `std`? Could we move it to itertools? That is possible, but observe that `filter`, `filter_map`, `find` and `find_map` together really form a complete table:
|||
|-------|---------|
| filter| find|
|filter_map|find_map|
It would be somewhat unsatisfying to have one quarter of this table live elsewhere :) Also, if `Itertools` adds an `find_map` method, it would be more difficult to move it to std due to name collision.
Hm, at this point I've searched for `filter_map` the umpteenth time, and, strangely, this time I do find this RFC: https://github.com/rust-lang/rfcs/issues/1801. I guess this could be an implementation though? :)
To sum up:
Pro:
- complete the symmetry with existing method
- codify a somewhat common non-obvious pattern
Contra:
- niche use case
- we can, and do, live without it
Move ascii::escape_default to libcore
As requested in #46409, the `ascii::escape_default` method has been added to the core library. All I did was copy over the `std::ascii` module file, remove the (redundant) `AsciiExt` trait, and change some of the documentation to match. None of the tests were changed.
I wasn't sure how to handle the annotations. For `EscapeDefault` and `escape_default()`, I changed them to `#[unstable(feature = "core_ascii", issue = "46409")]`. Is that alright? Or should I leave them as they were?
Required moving all fulldeps tests depending on `rand` to different locations as
now there's multiple `rand` crates that can't be implicitly linked against.
Add std/core::iter::repeat_with
Adds an iterator primitive `repeat_with` which is the "lazy" version of `repeat` but also more flexible since you can build up state with the `FnMut`. The design is mostly taken from `repeat`.
r? @rust-lang/libs
cc @withoutboats, @scottmcm
