d760aaf707
Ever plagued by #43081 the compiler can return surprising spans in situations related to procedural macros. This is exhibited by #47983 where whenever a procedural macro is invoked in a nested item context it would fail to have correct span information. While #43230 provided a "hack" to cache the token stream used for each item in the compiler it's not a full-blown solution. This commit continues to extend this "hack" a bit more to work for nested items. Previously in the parser the `parse_item` method would collect the tokens for an item into a cache on the item itself. It turned out, however, that nested items were parsed through the `parse_item_` method, so they didn't receive similar treatment. To remedy this situation the hook for collecting tokens was moved into `parse_item_` instead of `parse_item`. Afterwards the token collection scheme was updated to support nested collection of tokens. This is implemented by tracking `TokenStream` tokens instead of `TokenTree` to allow for collecting items into streams at intermediate layers and having them interleaved in the upper layers. All in all, this... Closes #47983 |
||
|---|---|---|
| .. | ||
| attr | ||
| diagnostics | ||
| ext | ||
| parse | ||
| util | ||
| ast.rs | ||
| build.rs | ||
| Cargo.toml | ||
| codemap.rs | ||
| config.rs | ||
| diagnostic_list.rs | ||
| entry.rs | ||
| feature_gate.rs | ||
| fold.rs | ||
| json.rs | ||
| lib.rs | ||
| ptr.rs | ||
| README.md | ||
| show_span.rs | ||
| std_inject.rs | ||
| str.rs | ||
| test.rs | ||
| test_snippet.rs | ||
| tokenstream.rs | ||
| visit.rs | ||
The syntax crate contains those things concerned purely with syntax
– that is, the AST ("abstract syntax tree"), parser, pretty-printer,
lexer, macro expander, and utilities for traversing ASTs.
For more information about how these things work in rustc, see the rustc guide: