Recover gracefully from `struct` parse errors
Currently the parser tries to recover from finding a keyword where a field name was expected, but this causes extra knock down parse errors that are completely irrelevant. Instead, bail out early in the parsing of the field and consume the remaining tokens in the block. This can reduce output significantly.
_Improvements based on the narrative in https://fasterthanli.me/articles/i-am-a-java-csharp-c-or-cplusplus-dev-time-to-do-some-rust_
Implement the `min_const_generics` feature gate
Implements both https://github.com/rust-lang/lang-team/issues/37 and https://github.com/rust-lang/compiler-team/issues/332.
Adds the new feature gate `#![feature(min_const_generics)]`.
This feature gate adds the following limitations to using const generics:
- generic parameters must only be used in types if they are trivial. (either `N` or `{ N }`)
- generic parameters must be either integers, `bool` or `char`.
We do allow arbitrary expressions in associated consts though, meaning that the following is allowed,
even if `<[u8; 0] as Foo>::ASSOC` is not const evaluatable.
```rust
trait Foo {
const ASSOC: usize;
}
impl<const N: usize> Foo for [u8; N] {
const ASSOC: usize = 64 / N;
}
```
r? @varkor cc @eddyb @withoutboats
Introduce NonterminalKind for more type-safe mbe parsing
It encapsulate the (part of) the interface between the parser and
macro by example (macro_rules) parser.
The second bit is somewhat more general `parse_ast_fragment`, which is
the reason why we keep some `parse_xxx` functions as public.
Deduplicate `::` -> `:` typo errors
Deduplicate errors caused by the same type ascription typo, including
ones suggested during parsing that would get reported again during
resolve. Fix#70382.
It encapsulate the (part of) the interface between the parser and
macro by example (macro_rules) parser.
The second bit is somewhat more general `parse_ast_fragment`, which is
the reason why we keep some `parse_xxx` functions as public.
* Deduplicate type ascription LHS errors
* Remove duplicated `:` -> `::` suggestion from parse error
* Tweak wording to be more accurate
* Modify `current_type_ascription` to reduce span wrangling
* remove now unnecessary match arm
* Add run-rustfix to appropriate tests
Handle case of incomplete local ty more gracefully
When encountering a local binding with a type that isn't completed, the
parser will reach a `=` token. When this happen, consider the type
"complete" as far as the parser is concerned to avoid further errors
being emitted by parse recovery logic.
When encountering a local binding with a type that isn't completed, the
parser will reach a `=` token. When this happen, consider the type
"complete" as far as the parser is concerned to avoid further errors
being emitted by parse recovery logic.
Accept tuple.0.0 as tuple indexing (take 2)
If we expect something identifier-like when parsing a field name after `.`, but encounter a float token, we break that float token into parts, similarly to how we break `&&` into `&` `&`, or `<<` into `<` `<`, etc.
An alternative to https://github.com/rust-lang/rust/pull/70420.
Tweak `::` -> `:` typo heuristic and reduce verbosity
Do not trigger on correct type ascription expressions with trailing
operators and _do_ trigger on likely path typos where a turbofish is
used.
On likely path typos, remove note explaining type ascription.
Clean up indentation.
r? @petrochenkov
Do not trigger on correct type ascription expressions with trailing
operators and _do_ trigger on likely path typos where a turbofish is
used.
On likely path typos, remove note explaining type ascription.
Audit hidden/short code suggestions
Should fix#73641.
Audit uses of `span_suggestion_short` and `tool_only_span_suggestion` (`span_suggestion_hidden` is already tested with `run-rustfix`). Leave some FIXMEs for futher improvements/fixes.
r? @estebank
Recover extra trailing angle brackets in struct definition
This commit applies the existing 'extra angle bracket recovery' logic
when parsing fields in struct definitions. This allows us to continue
parsing the struct's fields, avoiding spurious 'missing field' errors in
code that tries to use the struct.
expand: Stop using nonterminals for passing tokens to attribute and derive macros
Make one more step towards fully token-based expansion and fix issues described in https://github.com/rust-lang/rust/issues/72545#issuecomment-640276791.
Now `struct S;` is passed to `foo!(struct S;)` and `#[foo] struct S;` in the same way - as a token stream `struct S ;`, rather than a single non-terminal token `NtItem` which is then broken into parts later.
The cost is making pretty-printing of token streams less pretty.
Some of the pretty-printing regressions will be recovered by keeping jointness with each token, which we will need to do anyway.
Unfortunately, this is not exactly the same thing as https://github.com/rust-lang/rust/pull/73102.
One more observable effect is how `$crate` is printed in the attribute input.
Inside `NtItem` was printed as `crate` or `that_crate`, now as a part of a token stream it's printed as `$crate` (there are good reasons for these differences, see https://github.com/rust-lang/rust/pull/62393 and related PRs).
This may break old proc macros (custom derives) written before the main portion of the proc macro API (macros 1.2) was stabilized, those macros did `input.to_string()` and reparsed the result, now that result can contain `$crate` which cannot be reparsed.
So, I think we should do this regardless, but we need to run crater first.
r? @Aaron1011
This commit applies the existing 'extra angle bracket recovery' logic
when parsing fields in struct definitions. This allows us to continue
parsing the struct's fields, avoiding spurious 'missing field' errors in
code that tries to use the struct.
Record span of `const` kw in GenericParamKind
Context: this is needed for a fix of https://github.com/rust-lang/rustfmt/issues/4263,
which currently records the span of a const generic param incorrectly
because the location of the `const` kw is not known.
I am not sure how to add tests for this; any guidance in how to do so
would be appreciated 🙂
proc_macro: Stop flattening groups with dummy spans
Reduce the scope of the hack described in https://github.com/rust-lang/rust/issues/72545#issuecomment-640276791.
We still pass AST fragments to attribute and derive macros as single nonterminal tokens rather than as tokens streams, but now use a precise flag instead of the span-based heuristic that could do lead to incorrect behavior in unrelated cases.
https://github.com/rust-lang/rust/pull/73345 attempts to fully resolve this issue, but there are some compatibility issues to be addressed.
Always capture tokens for `macro_rules!` arguments
When we invoke a proc-macro, the `TokenStream` we pass to it may contain 'interpolated' AST fragments, represented by `rustc_ast::token::Nonterminal`. In order to correctly, pass a `Nonterminal` to a proc-macro, we need to have 'captured' its `TokenStream` at the time the AST was parsed.
Currently, we perform this capturing when attributes are present on items and expressions, since we will end up using a `Nonterminal` to pass the item/expr to any proc-macro attributes it is annotated with. However, `Nonterminal`s are also introduced by the expansion of metavariables in `macro_rules!` macros. Since these metavariables may be passed to proc-macros, we need to have tokens available to avoid the need to pretty-print and reparse (see https://github.com/rust-lang/rust/issues/43081).
This PR unconditionally performs token capturing for AST items and expressions that are passed to a `macro_rules!` invocation. We cannot know in advance if captured item/expr will be passed to proc-macro, so this is needed to ensure that tokens will always be available when they are needed.
This ensures that proc-macros will receive tokens with proper `Spans` (both location and hygiene) in more cases. Like all work on https://github.com/rust-lang/rust/issues/43081, this will cause regressions in proc-macros that were relying on receiving tokens with dummy spans.
In this case, Crater revealed only one regression: the [Pear](https://github.com/SergioBenitez/Pear) crate (a helper for [rocket](https://github.com/SergioBenitez/Rocket)), which was previously [fixed](https://github.com/SergioBenitez/Pear/pull/25) as part of https://github.com/rust-lang/rust/pull/73084.
This regression manifests itself as the following error:
```
[INFO] [stdout] error: proc macro panicked
[INFO] [stdout] --> /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/rocket_http-0.4.5/src/parse/uri/parser.rs:119:34
[INFO] [stdout] |
[INFO] [stdout] 119 | let path_and_query = pear_try!(path_and_query(is_pchar));
[INFO] [stdout] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] |
[INFO] [stdout] = help: message: called `Option::unwrap()` on a `None` value
[INFO] [stdout] = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
```
It can be fixed by running `cargo update -p pear`, which updates your `Cargo.lock` to use the latest version of Pear (which includes a bugfix for the regression).
Split out from https://github.com/rust-lang/rust/pull/73084/
Context: this is needed to fix https://github.com/rust-lang/rustfmt/issues/4263,
which currently records the span of a const generic param incorrectly
because the location of the `const` kw is not known.
I am not sure how to add tests for this; any guidance in how to do so
would be appreciated 🙂
