Addressed #51602Fixed#51602
r? @estebank
here I have addressed the case where `in` was not expected right after `if` block. Speaking of `type ascription` I am not sure if this the best approach which I have implemented. Plus I think one more test case can be added to test `type-ascription` case, though I don't have any at this point of time. I will ping you again if all existing testcases pass.
A few cleanups and minor improvements for the lexer
- improve readability by adjusting the formatting of some function signatures and adding some newlines
- reorder some functions for easier reading
- remove redundant `'static` in `const`s
- remove some explicit `return`s
- read directly to a `String` in `gather_comments_and_literals`
- change `unwrap_or!` (macro) to `unwrap_or` (function)
- move an `assert!`ion from `try_next_token` (called in a loop) to `try_real_token` after all calls to `try_next_token`
- `#[inline]` some one-liner functions
- assign directly from an `if-else` expression
- refactor a `match` to `map_or`
- add a `token::is_irrelevant` function to detect tokens that are not "`real`"
The error and check for this already existed, but the parser didn't try to parse trait method arguments as patterns, so the error was never emitted. This surfaces the error, so we get better errors than simple parse errors.
Replace push loops with extend() where possible
Or set the vector capacity where I couldn't do it.
According to my [simple benchmark](https://gist.github.com/ljedrz/568e97621b749849684c1da71c27dceb) `extend`ing a vector can be over **10 times** faster than `push`ing to it in a loop:
10 elements (6.1 times faster):
```
test bench_extension ... bench: 75 ns/iter (+/- 23)
test bench_push_loop ... bench: 458 ns/iter (+/- 142)
```
100 elements (11.12 times faster):
```
test bench_extension ... bench: 87 ns/iter (+/- 26)
test bench_push_loop ... bench: 968 ns/iter (+/- 3,528)
```
1000 elements (11.04 times faster):
```
test bench_extension ... bench: 311 ns/iter (+/- 9)
test bench_push_loop ... bench: 3,436 ns/iter (+/- 233)
```
Seems like a good idea to use `extend` as much as possible.
Ever plagued by #43081 the compiler can return surprising spans in situations
related to procedural macros. This is exhibited by #47983 where whenever a
procedural macro is invoked in a nested item context it would fail to have
correct span information.
While #43230 provided a "hack" to cache the token stream used for each item in
the compiler it's not a full-blown solution. This commit continues to extend
this "hack" a bit more to work for nested items.
Previously in the parser the `parse_item` method would collect the tokens for an
item into a cache on the item itself. It turned out, however, that nested items
were parsed through the `parse_item_` method, so they didn't receive similar
treatment. To remedy this situation the hook for collecting tokens was moved
into `parse_item_` instead of `parse_item`.
Afterwards the token collection scheme was updated to support nested collection
of tokens. This is implemented by tracking `TokenStream` tokens instead of
`TokenTree` to allow for collecting items into streams at intermediate layers
and having them interleaved in the upper layers.
All in all, this...
Closes#47983
Prepare proc_macro for decoupling it from the rest of the compiler.
This is #49219 up to the point where the bridge is introduced. Aside from moving some code around, the largest change is the rewrite of `proc_macro::quote` to be simpler and do less introspection.
I'd like to also extend `quote!` with `${stmt;...;expr}` instead of just `$variable` (and maybe even `$(... $iter ...)*`), which seems pretty straight-forward now, but I don't know if/when I should.
r? @alexcrichton or @dtolnay cc @jseyfried @petrochenkov
proc_macro: Preserve spans of attributes on functions
This commit updates the tokenization of items which are subsequently passed to
`proc_macro` to ensure that span information is preserved on attributes as much
as possible. Previously this area of the code suffered from #43081 where we
haven't actually implemented converting an attribute to to a token tree yet, but
a local fix was possible here.
Closes#47941
This commit updates the tokenization of items which are subsequently passed to
`proc_macro` to ensure that span information is preserved on attributes as much
as possible. Previously this area of the code suffered from #43081 where we
haven't actually implemented converting an attribute to to a token tree yet, but
a local fix was possible here.
Closes#47941
clarify why we're suggesting removing semicolon after braced items
Previously (issue #46186, pull-request #46258), a suggestion was added
to remove the semicolon after we fail to parse an item, but issue #51603
complains that it's still insufficiently obvious why. Let's add a note.
Resolves#51603.
