Improve documentation for built-in macros
This is the `libcore` part of https://github.com/rust-lang/rust/pull/62086.
Right now the only effect is improved documentation.
The changes in the last few commits are required to make the `libcore` change compile successfully.
Refactor unicode.py script
Hi, I noticed that the `unicode.py` script used some deprecated escapes in regular expressions. E.g. `\d`, `\w`, `\.` will be illegal in the future without "raw strings". This is now fixed. I have also cleaned up the script quite a bit.
## Escape deprecation
OK (note the `r`):
`re.compile(r"\d")`
Deprecated (from Python 3.6 onwards, see [here][link1] and [here][link2]):
`re.compile("\d")`.
[link1]: https://docs.python.org/3.6/whatsnew/3.6.html#deprecated-python-behavior
[link2]: https://bugs.python.org/issue27364
This was evident running the script using Python 3.7 like so:
```
$ python3 -Wall unicode.py
unicode.py:227: DeprecationWarning: invalid escape sequence \w
re1 = re.compile("^ *([0-9A-F]+) *; *(\w+)")
unicode.py:228: DeprecationWarning: invalid escape sequence \.
re2 = re.compile("^ *([0-9A-F]+)\.\.([0-9A-F]+) *; *(\w+)")
unicode.py:453: DeprecationWarning: invalid escape sequence \d
pattern = "for Version (\d+)\.(\d+)\.(\d+) of the Unicode"
```
The documentation states that
> A backslash-character pair that is not a valid escape sequence now generates a DeprecationWarning. Although this will eventually become a SyntaxError, that will not be for several Python releases.
## Testing
To test my changes, I had to add support for choosing the Unicode version to use. The script will default to latest release (which is 12.0.0 at the moment, repo has 11.0.0 checked in).
The script generates the exact same output for version 11.0.0 with Python 2.7 and 3.7 and no longer generates any deprecation warnings:
```
$ python3 -Wall unicode.py -v 11.0.0
Using Unicode version: 11.0.0
Regenerated tables.rs.
$ git diff tables.rs
$ python2 -Wall unicode.py -v 11.0.0
Using Unicode version: 11.0.0
Regenerated tables.rs.
$ git diff tables.rs
$ python2 --version
Python 2.7.16
$ python3 --version
Python 3.7.3
```
## Extra functionality
Furthermore, the script will check and download the latest Unicode version by default (without the `-v` argument). The `--help` is below:
```
$ ./unicode.py --help
usage: unicode.py [-h] [-v VERSION]
Regenerate Unicode tables (tables.rs).
optional arguments:
-h, --help show this help message and exit
-v VERSION, --version VERSION
Unicode version to use (if not specified, defaults to
latest available final release).
```
## Cleanups
I have cleaned up the code quite a bit, with Python best practices and code style in mind. I'm happy to provide more details and rationale for all my changes if the reviewers so desire.
One externally visible change is that the Unicode data will now be downloaded into `src/libcore/unicode/downloaded` directory suffixed by Unicode version:
```
$ pwd
.../rust/src/libcore/unicode
$ exa -T downloaded/
downloaded
├── 11.0.0
│ ├── DerivedCoreProperties.txt
│ ├── DerivedNormalizationProps.txt
│ ├── PropList.txt
│ ├── ReadMe.txt
│ ├── Scripts.txt
│ ├── SpecialCasing.txt
│ └── UnicodeData.txt
└── 12.0.0
├── DerivedCoreProperties.txt
├── DerivedNormalizationProps.txt
├── PropList.txt
├── ReadMe.txt
├── Scripts.txt
├── SpecialCasing.txt
└── UnicodeData.txt
```
When possible without changing semantics, implement Iterator::last in terms of DoubleEndedIterator::next_back for types in liballoc and libcore.
Provided that the iterator has finite length and does not trigger user-provided code, this is safe.
What follows is a full list of the DoubleEndedIterators in liballoc/libcore and whether this optimization is safe, and if not, why not.
src/liballoc/boxed.rs
Box: Pass through to avoid defeating optimization of the underlying DoubleIterator implementation. This has no correctness impact.
src/liballoc/collections/binary_heap.rs
Iter: Pass through to avoid defeating optimizations on slice::Iter
IntoIter: Not safe, changes Drop order
Drain: Not safe, changes Drop order
src/liballoc/collections/btree/map.rs
Iter: Safe to call next_back, invokes no user defined code.
IterMut: ditto
IntoIter: Not safe, changes Drop order
Keys: Safe to call next_back, invokes no user defined code.
Values: ditto
ValuesMut: ditto
Range: ditto
RangeMut: ditto
src/liballoc/collections/btree/set.rs
Iter: Safe to call next_back, invokes no user defined code.
IntoIter: Not safe, changes Drop order
Range: Safe to call next_back, invokes no user defined code.
src/liballoc/collections/linked_list.rs
Iter: Safe to call next_back, invokes no user defined code.
IterMut: ditto
IntoIter: Not safe, changes Drop order
src/liballoc/collections/vec_deque.rs
Iter: Safe to call next_back, invokes no user defined code.
IterMut: ditto
IntoIter: Not safe, changes Drop order
Drain: ditto
src/liballoc/string.rs
Drain: Safe because return type is a primitive (char)
src/liballoc/vec.rs
IntoIter: Not safe, changes Drop order
Drain: ditto
Splice: ditto
src/libcore/ascii.rs
EscapeDefault: Safe because return type is a primitive (u8)
src/libcore/iter/adapters/chain.rs
Chain: Not safe, invokes user defined code (Iterator impl)
src/libcore/iter/adapters/flatten.rs
FlatMap: Not safe, invokes user defined code (Iterator impl)
Flatten: ditto
FlattenCompat: ditto
src/libcore/iter/adapters/mod.rs
Rev: Not safe, invokes user defined code (Iterator impl)
Copied: ditto
Cloned: Not safe, invokes user defined code (Iterator impl and T::clone)
Map: Not safe, invokes user defined code (Iterator impl + closure)
Filter: ditto
FilterMap: ditto
Enumerate: Not safe, invokes user defined code (Iterator impl)
Skip: ditto
Fuse: ditto
Inspect: ditto
src/libcore/iter/adapters/zip.rs
Zip: Not safe, invokes user defined code (Iterator impl)
src/libcore/iter/range.rs
ops::Range: Not safe, changes Drop order, but ALREADY HAS SPECIALIZATION
ops::RangeInclusive: ditto
src/libcore/iter/sources.rs
Repeat: Not safe, calling last should iloop.
Empty: No point, iterator is at most one item long.
Once: ditto
OnceWith: ditto
src/libcore/option.rs
Item: No point, iterator is at most one item long.
Iter: ditto
IterMut: ditto
IntoIter: ditto
src/libcore/result.rs
Iter: No point, iterator is at most one item long
IterMut: ditto
IntoIter: ditto
src/libcore/slice/mod.rs
Split: Not safe, invokes user defined closure
SplitMut: ditto
RSplit: ditto
RSplitMut: ditto
Windows: Safe, already has specialization
Chunks: ditto
ChunksMut: ditto
ChunksExact: ditto
ChunksExactMut: ditto
RChunks: ditto
RChunksMut: ditto
RChunksExact: ditto
RChunksExactMut: ditto
src/libcore/str/mod.rs
Chars: Safe, already has specialization
CharIndices: ditto
Bytes: ditto
Lines: Safe to call next_back, invokes no user defined code.
LinesAny: Deprecated
Everything that is generic over P: Pattern: Not safe because Pattern invokes user defined code.
SplitWhitespace: Safe to call next_back, invokes no user defined code.
SplitAsciiWhitespace: ditto
This is attempt 2 of #60130.
r? @sfackler
nth_back for chunks_exact
wip nth_back for chunks_exact
working nth_back for chunks exact
Signed-off-by: wizAmit <amitforfriends_dns@yahoo.com>
r? @timvermeulen
r? @scottmcm
Provided that the iterator has finite length and does not trigger user-provided code, this is safe.
What follows is a full list of the DoubleEndedIterators in liballoc/libcore and whether this optimization is safe, and if not, why not.
src/liballoc/boxed.rs
Box: Pass through to avoid defeating optimization of the underlying DoubleIterator implementation. This has no correctness impact.
src/liballoc/collections/binary_heap.rs
Iter: Pass through to avoid defeating optimizations on slice::Iter
IntoIter: Not safe, changes Drop order
Drain: Not safe, changes Drop order
src/liballoc/collections/btree/map.rs
Iter: Safe to call next_back, invokes no user defined code.
IterMut: ditto
IntoIter: Not safe, changes Drop order
Keys: Safe to call next_back, invokes no user defined code.
Values: ditto
ValuesMut: ditto
Range: ditto
RangeMut: ditto
src/liballoc/collections/btree/set.rs
Iter: Safe to call next_back, invokes no user defined code.
IntoIter: Not safe, changes Drop order
Range: Safe to call next_back, invokes no user defined code.
src/liballoc/collections/linked_list.rs
Iter: Safe to call next_back, invokes no user defined code.
IterMut: ditto
IntoIter: Not safe, changes Drop order
src/liballoc/collections/vec_deque.rs
Iter: Safe to call next_back, invokes no user defined code.
IterMut: ditto
IntoIter: Not safe, changes Drop order
Drain: ditto
src/liballoc/string.rs
Drain: Safe because return type is a primitive (char)
src/liballoc/vec.rs
IntoIter: Not safe, changes Drop order
Drain: ditto
Splice: ditto
src/libcore/ascii.rs
EscapeDefault: Safe because return type is a primitive (u8)
src/libcore/iter/adapters/chain.rs
Chain: Not safe, invokes user defined code (Iterator impl)
src/libcore/iter/adapters/flatten.rs
FlatMap: Not safe, invokes user defined code (Iterator impl)
Flatten: ditto
FlattenCompat: ditto
src/libcore/iter/adapters/mod.rs
Rev: Not safe, invokes user defined code (Iterator impl)
Copied: ditto
Cloned: Not safe, invokes user defined code (Iterator impl and T::clone)
Map: Not safe, invokes user defined code (Iterator impl + closure)
Filter: ditto
FilterMap: ditto
Enumerate: Not safe, invokes user defined code (Iterator impl)
Skip: ditto
Fuse: ditto
Inspect: ditto
src/libcore/iter/adapters/zip.rs
Zip: Not safe, invokes user defined code (Iterator impl)
src/libcore/iter/range.rs
ops::Range: Not safe, changes Drop order, but ALREADY HAS SPECIALIZATION
ops::RangeInclusive: ditto
src/libcore/iter/sources.rs
Repeat: Not safe, calling last should iloop.
Empty: No point, iterator is at most one item long.
Once: ditto
OnceWith: ditto
src/libcore/option.rs
Item: No point, iterator is at most one item long.
Iter: ditto
IterMut: ditto
IntoIter: ditto
src/libcore/result.rs
Iter: No point, iterator is at most one item long
IterMut: ditto
IntoIter: ditto
src/libcore/slice/mod.rs
Split: Not safe, invokes user defined closure
SplitMut: ditto
RSplit: ditto
RSplitMut: ditto
Windows: Safe, already has specialization
Chunks: ditto
ChunksMut: ditto
ChunksExact: ditto
ChunksExactMut: ditto
RChunks: ditto
RChunksMut: ditto
RChunksExact: ditto
RChunksExactMut: ditto
src/libcore/str/mod.rs
Chars: Safe, already has specialization
CharIndices: ditto
Bytes: ditto
Lines: Safe to call next_back, invokes no user defined code.
LinesAny: Deprecated
Everything that is generic over P: Pattern: Not safe because Pattern invokes user defined code.
SplitWhitespace: Safe to call next_back, invokes no user defined code.
SplitAsciiWhitespace: ditto
Revert "Set test flag when rustdoc is running with --test option"
Reverts https://github.com/rust-lang/rust/pull/59940.
It caused doctests in this repository to no longer be tested including all of the core crate.
Add suggestion for missing `.await` keyword
This commit adds a suggestion diagnostic for missing `.await`. In order to do this, the trait `Future` is promoted to a lang item.
Compiling code of the form:
```rust
#![feature(async_await)]
fn take_u32(x: u32) {}
async fn make_u32() -> u32 {
22
}
async fn foo() {
let x = make_u32();
take_u32(x)
}
fn main() {}
```
Will now result in the suggestion:
```
error[E0308]: mismatched types
--> src/main.rs:11:18
|
11 | take_u32(x)
| ^
| |
| expected u32, found opaque type
| help: consider using `.await` here: `x.await`
|
= note: expected type `u32`
found type `impl std::future::Future`
```
This commit does not cover chained expressions and therefore only covers the case originally pointed out in #61076. Cases I can think of that still need to be covered:
- [ ] Return places for functions
- [ ] Field access
- [ ] Method invocation
I'm planning to submit PRs for each of these separately as and when I have figured them out.
Fix meta-variable binding errors in macros
The errors are either:
- The meta-variable used in the right-hand side is not bound (or defined) in the
left-hand side.
- The meta-variable used in the right-hand side does not repeat with the same
kleene operator as its binder in the left-hand side. Either it does not repeat
enough, or it uses a different operator somewhere.
This change should have no semantic impact.
Found by https://github.com/rust-lang/rust/pull/62008
The errors are either:
- The meta-variable used in the right-hand side is not bound (or defined) in the
left-hand side.
- The meta-variable used in the right-hand side does not repeat with the same
kleene operator as its binder in the left-hand side. Either it does not repeat
enough, or it uses a different operator somewhere.
This change should have no semantic impact.
Remove the default type of `Rem::Output`
Associated type defaults are not yet stable, and `Rem` is the only trait that specifies a default. Let's see what breaks when it's removed.
cc https://github.com/rust-lang/rust/pull/61812#issuecomment-502394566
cc @Centril
@bors try
