parser: Split `+=` into `+` and `=` where `+` is explicitly requested (such as generics)
Added functions in tokens to check whether a token leads with `+`. Used them when parsing to allow for token splitting of `+=` into `+` and `=`.
Fixes https://github.com/rust-lang/rust/issues/47856
Accept `..` in incorrect position to avoid further errors
We currently give a specific message when encountering a `..` anywhere
other than the end of a pattern. Modify the parser to accept it (while
still emitting the error) so that we don't also trigger "missing fields
in pattern" errors afterwards.
Add suggestions to either remove trailing `,` or moving the `..` to the
end.
Follow up to #49268.
When using `..` somewhere other than the end, parse the rest of the
pattern correctly while still emitting an error.
Add suggestions to either remove trailing `,` or moving the `..` to the
end.
We currently give a specific message when encountering a `..` anywhere
other than the end of a pattern. Modify the parser to accept it (while
still emitting the error) so that we don't also trigger "missing fields
in pattern" errors afterwards.
This commit is concerned with the case where the user tries to mutably
borrow a mutable reference, thereby triggering an error. Instead of the
existing suggestion to make the binding mutable, the compiler will now
suggest to avoid borrowing altogether.
add suggestion applicabilities to librustc and libsyntax
A down payment on #50723. Interested in feedback on whether my `MaybeIncorrect` vs. `MachineApplicable` judgement calls are well-calibrated (and that we have a consensus on what this means).
r? @Manishearth
cc @killercup @estebank
The `FatalError.raise()` might seem unmotivated (in most places in
the compiler, `err.emit()` suffices), but it's actually used to
maintain behavior (viz., stop lexing, don't emit potentially spurious
errors looking for the next token after the bad Unicodepoint in the
exponent): the previous revision's `self.err_span_` ultimately calls
`Handler::emit`, which aborts if the `Handler`'s continue_after_error
flag is set, which seems to typically be true during lexing (see
`phase_1_parse_input` and and how `CompileController::basic` has
`continue_parse_after_error: false` in librustc_driver).
Also, let's avoid apostrophes in error messages (the present author
would argue that users expect a reassuringly detached, formal,
above-it-all tone from a Serious tool like a compiler), and use an
RLS-friendly structured suggestion.
Resolves#49746.
Fix span for type-only arguments
Currently it points to the comma or parenthesis before the type, which is broken
cc @mark-i-m this is what broke #48309
r? @estebank
rustc: Correctly pretty-print macro delimiters
This commit updates the `Mac_` AST structure to keep track of the delimiters
that it originally had for its invocation. This allows us to faithfully
pretty-print macro invocations not using parentheses (e.g. `vec![...]`). This in
turn helps procedural macros due to #43081.
Closes#50840
This commit updates the `Mac_` AST structure to keep track of the delimiters
that it originally had for its invocation. This allows us to faithfully
pretty-print macro invocations not using parentheses (e.g. `vec![...]`). This in
turn helps procedural macros due to #43081.
Closes#50840
rustc: Fix procedural macros generating lifetime tokens
This commit fixes an accidental regression from #50473 where lifetime tokens
produced by procedural macros ended up getting lost in translation in the
compiler and not actually producing parseable code. The issue lies in the fact
that a lifetime's `Ident` is prefixed with `'`. The `glue` implementation for
gluing joint tokens together forgot to take this into account so the lifetime
inside of `Ident` was missing the leading tick!
The `glue` implementation here is updated to create a new `Symbol` in these
situations to manufacture a new `Ident` with a leading tick to ensure it parses
correctly.
Closes#50942
rustc: Fix joint-ness of stringified token-streams
This commit fixes `StringReader`'s parsing of tokens which have been stringified
through procedural macros. Whether or not a token tree is joint is defined by
span information, but when working with procedural macros these spans are often
dummy and/or overridden which means that they end up considering all operators
joint if they can!
The fix here is to track the raw source span as opposed to the overridden span.
With this information we can more accurately classify `Punct` structs as either
joint or not.
Closes#50700
This commit fixes an accidental regression from #50473 where lifetime tokens
produced by procedural macros ended up getting lost in translation in the
compiler and not actually producing parseable code. The issue lies in the fact
that a lifetime's `Ident` is prefixed with `'`. The `glue` implementation for
gluing joint tokens together forgot to take this into account so the lifetime
inside of `Ident` was missing the leading tick!
The `glue` implementation here is updated to create a new `Symbol` in these
situations to manufacture a new `Ident` with a leading tick to ensure it parses
correctly.
Closes#50942
lexer: Fix span override for the first token in a string
Previously due to peculiarities of `StringReader` construction something like `"a b c d".parse::<TokenStream>()` gave you one non-overridden span for `a` and then three correctly overridden spans for `b`, `c` and `d`.
Now all the spans are overridden.
rustc: introduce {ast,hir}::AnonConst to consolidate so-called "embedded constants".
Previously, constants in array lengths and enum variant discriminants were "merely an expression", and had no separate ID for, e.g. type-checking or const-eval, instead reusing the expression's.
That complicated code working with bodies, because such constants were the only special case where the "owner" of the body wasn't the HIR parent, but rather the same node as the body itself.
Also, if the body happened to be a closure, we had no way to allocate a `DefId` for both the constant *and* the closure, leading to *several* bugs (mostly ICEs where type errors were expected).
This PR rectifies the situation by adding another (`{ast,hir}::AnonConst`) node around every such constant. Also, const generics are expected to rely on the new `AnonConst` nodes, as well (cc @varkor).
* fixes#48838
* fixes#50600
* fixes#50688
* fixes#50689
* obsoletes #50623
r? @nikomatsakis
Consider this a down payment on #50723. To recap, an `Applicability`
enum was recently (#50204) added, to convey to Rustfix and other tools
whether we think it's OK for them to blindly apply the suggestion, or
whether to prompt a human for guidance (because the suggestion might
contain placeholders that we can't infer, or because we think it has a
sufficiently high probability of being wrong even though it's—
presumably—right often enough to be worth emitting in the first place).
When a suggestion is marked as `MaybeIncorrect`, we try to use comments
to indicate precisely why (although there are a few places where we just
say `// speculative` because the present author's subjective judgement
balked at the idea that the suggestion has no false positives).
The `run-rustfix` directive is opporunistically set on some relevant UI
tests (and a couple tests that were in the `test/ui/suggestions`
directory, even if the suggestions didn't originate in librustc or
libsyntax). This is less trivial than it sounds, because a surprising
number of test files aren't equipped to be tested as fixed even when
they contain successfully fixable errors, because, e.g., there are more,
not-directly-related errors after fixing. Some test files need an
attribute or underscore to avoid unused warnings tripping up the "fixed
code is still producing diagnostics" check despite the fixes being
correct; this is an interesting contrast-to/inconsistency-with the
behavior of UI tests (which secretly pass `-A unused`), a behavior which
we probably ought to resolve one way or the other (filed issue #50926).
A few suggestion labels are reworded (e.g., to avoid phrasing it as a
question, which which is discouraged by the style guidelines listed in
`.span_suggestion`'s doc-comment).
Speed up the macro parser
These three commits reduce the number of allocations done by the macro parser, in some cases dramatically. For example, for a clean check builds of html5ever, the number of allocations is reduced by 40%.
Here are the rustc-benchmarks that are sped up by at least 1%.
```
html5ever-check
avg: -6.6% min: -10.3% max: -4.1%
html5ever
avg: -5.2% min: -9.5% max: -2.8%
html5ever-opt
avg: -4.3% min: -9.3% max: -1.6%
crates.io-check
avg: -1.8% min: -2.9% max: -0.6%
crates.io-opt
avg: -1.0% min: -2.2% max: -0.1%
crates.io
avg: -1.1% min: -2.2% max: -0.2%
```
This commit fixes `StringReader`'s parsing of tokens which have been stringified
through procedural macros. Whether or not a token tree is joint is defined by
span information, but when working with procedural macros these spans are often
dummy and/or overridden which means that they end up considering all operators
joint if they can!
The fix here is to track the raw source span as opposed to the overridden span.
With this information we can more accurately classify `Punct` structs as either
joint or not.
Closes#50700
Implement edition hygiene for keywords
Determine "keywordness" of an identifier in its hygienic context.
cc https://github.com/rust-lang/rust/pull/49611
I've resurrected `proc` as an Edition-2015-only keyword for testing purposes, but it should probably be buried again. EDIT: `proc` is removed again.
