Remove support for 1-token lookahead from the lexer
`StringReader` maintained `peek_token` and `peek_span_src_raw` for look ahead.
`peek_token` was used only by rustdoc syntax coloring. After moving peeking logic into highlighter, I was able to remove `peek_token` from the lexer. I tried to use `iter::Peekable`, but that wasn't as pretty as I hoped, due to buffered fatal errors. So I went with hand-rolled peeking.
After that I've noticed that the only peeking behavior left was for raw tokens to test tt jointness. I've rewritten it in terms of trivia tokens, and not just spans.
After that it became possible to simplify the awkward constructor of the lexer, which could return `Err` if the first peeked token contained error.
Remove io::Result from syntax::print
Since we're now writing directly to the vector, there's no need to
thread results through the whole printing infrastructure
The reader itself doesn't need ability to peek tokens, so it's better
if clients implement this functionality.
This hopefully becomes especially easy once we use iterator interface
for lexer, but this is not too easy at the moment, because of buffered
errors.
Remove `ast::Guard`
With the introduction of `ast::ExprKind::Let` in https://github.com/rust-lang/rust/pull/60861, the `ast::Guard` structure is now redundant in terms of representing [`if let` guards](https://github.com/rust-lang/rust/issues/51114) in AST since it can be represented by `ExprKind::Let` syntactically. Therefore, we remove `ast::Guard` here.
However, we keep `hir::Guard` because the semantic representation is a different matter and this story is more unclear right now (might involve `goto 'arm` in HIR or something...).
r? @petrochenkov
Special-case literals in `parse_bottom_expr`.
This makes parsing faster, particularly for code with large constants,
for two reasons:
- it skips all the keyword comparisons for literals;
- it skips the allocation done by the `mk_expr` call in
`parse_literal_maybe_minus`.
r? @petrochenkov
Allow attributes in formal function parameters
Implements https://github.com/rust-lang/rust/issues/60406.
This is my first contribution to the compiler and since this is a large and complex project, I am not fully aware of the consequences of the changes I have made.
**TODO**
- [x] Forbid some built-in attributes.
- [x] Expand cfg/cfg_attr
lexer: Disallow bare CR in raw byte strings
Handles bare CR ~but doesn't translate `\r\n` to `\n` yet in raw strings yet~ and translates CRLF to LF in raw strings.
As a side-note I think it'd be good to change the `unescape_` to return plain iterators to reduce some boilerplate (e.g. `has_error` could benefit from collecting `Result<T>` and aborting early on errors) but will do that separately, unless I missed something here that prevents it.
@matklad @petrochenkov thoughts?
This makes parsing faster, particularly for code with large constants,
for two reasons:
- it skips all the keyword comparisons for literals;
- it replaces the unnecessary `parse_literal_maybe_minus` call with
`parse_lit`, avoiding an unnecessary allocation via `mk_expr`.
