It's present within `Token::Interpolated` as an optimization, so that if
a nonterminal is converted to a `TokenStream` multiple times, the
first-computed value is saved and reused.
But in practice it's not needed. `interpolated_to_tokenstream()` is a
cold function: it's only called a few dozen times while compiling rustc
itself, and a few hundred times across the entire `rustc-perf` suite.
Furthermore, when it is called, it is almost always the first
conversion, so no benefit is gained from it.
So this commit removes `LazyTokenStream`, along with the now-unnecessary
`Token::interpolated()`.
As well as a significant simplification, the removal speeds things up
slightly, mostly due to not having to `drop` the `LazyTokenStream`
instances.
Rename rustc_errors dependency in rust 2018 crates
I think this is a better solution than `use rustc_errors as errors` in `lib.rs` and `use crate::errors` in modules.
Related: rust-lang/cargo#5653
cc #58099
r? @Centril
This commit changes `syntax::fold::Folder` from a functional style
(where most methods take a `T` and produce a new `T`) to a more
imperative style (where most methods take and modify a `&mut T`), and
renames it `syntax::mut_visit::MutVisitor`.
The first benefit is speed. The functional style does not require any
reallocations, due to the use of `P::map` and
`MoveMap::move_{,flat_}map`. However, every field in the AST must be
overwritten; even those fields that are unchanged are overwritten with
the same value. This causes a lot of unnecessary memory writes. The
imperative style reduces instruction counts by 1--3% across a wide range
of workloads, particularly incremental workloads.
The second benefit is conciseness; the imperative style is usually more
concise. E.g. compare the old functional style:
```
fn fold_abc(&mut self, abc: ABC) {
ABC {
a: fold_a(abc.a),
b: fold_b(abc.b),
c: abc.c,
}
}
```
with the imperative style:
```
fn visit_abc(&mut self, ABC { a, b, c: _ }: &mut ABC) {
visit_a(a);
visit_b(b);
}
```
(The reductions get larger in more complex examples.)
Overall, the patch removes over 200 lines of code -- even though the new
code has more comments -- and a lot of the remaining lines have fewer
characters.
Some notes:
- The old style used methods called `fold_*`. The new style mostly uses
methods called `visit_*`, but there are a few methods that map a `T`
to something other than a `T`, which are called `flat_map_*` (`T` maps
to multiple `T`s) or `filter_map_*` (`T` maps to 0 or 1 `T`s).
- `move_map.rs`/`MoveMap`/`move_map`/`move_flat_map` are renamed
`map_in_place.rs`/`MapInPlace`/`map_in_place`/`flat_map_in_place` to
reflect their slightly changed signatures.
- Although this commit renames the `fold` module as `mut_visit`, it
keeps it in the `fold.rs` file, so as not to confuse git. The next
commit will rename the file.
Simplify `TokenStream` some more
These commits simplify `TokenStream`, remove `ThinTokenStream`, and avoid some clones. The end result is simpler code and a slight perf win on some benchmarks.
r? @petrochenkov
Implement basic input validation for built-in attributes
Correct top-level shape (`#[attr]` vs `#[attr(...)]` vs `#[attr = ...]`) is enforced for built-in attributes, built-in attributes must also fit into the "meta-item" syntax (aka the "classic attribute syntax").
For some subset of attributes (found by crater run), errors are lowered to deprecation warnings.
NOTE: This PR previously included https://github.com/rust-lang/rust/pull/57367 as well.
`TokenStream` is currently recursive in *two* ways:
- the `TokenTree` variant contains a `ThinTokenStream`, which can
contain a `TokenStream`;
- the `TokenStream` variant contains a `Vec<TokenStream>`.
The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.
The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)
Overall, the commit reduces the number of lines of code by almost 200.
Because it's an extra type layer that doesn't really help; in a couple
of places it actively gets in the way, and overall removing it makes the
code nicer. It does, however, move `tokenstream::TokenTree` further away
from the `TokenTree` in `quote.rs`.
More importantly, this change reduces the size of `TokenStream` from 48
bytes to 40 bytes on x86-64, which is enough to slightly reduce
instruction counts on numerous benchmarks, the best by 1.5%.
Note that `open_tt` and `close_tt` have gone from being methods on
`Delimited` to associated methods of `TokenTree`.
proc_macro::Group::span_open and span_close
Before this addition, every delimited group like `(`...`)` `[`...`]` `{`...`}` has only a single Span that covers the full source location from opening delimiter to closing delimiter. This makes it impossible for a procedural macro to trigger an error pointing to just the opening or closing delimiter. The Rust compiler does not seem to have the same limitation:
```rust
mod m {
type T =
}
```
```console
error: expected type, found `}`
--> src/main.rs:3:1
|
3 | }
| ^
```
On that same input, a procedural macro would be forced to trigger the error on the last token inside the block, on the entire block, or on the next token after the block, none of which is really what you want for an error like above.
This commit adds `group.span_open()` and `group.span_close()` which access the Span associated with just the opening delimiter and just the closing delimiter of the group. Relevant to Syn as we implement real error messages for when parsing fails in a procedural macro: https://github.com/dtolnay/syn/issues/476.
```diff
impl Group {
fn span(&self) -> Span;
+ fn span_open(&self) -> Span;
+ fn span_close(&self) -> Span;
}
```
Fixes#48187
r? @alexcrichton
resolve: Implement prelude search for macro paths, implement tool attributes
When identifier is macro path is resolved in scopes (i.e. the first path segment - `foo` in `foo::mac!()` or `foo!()`), scopes are searched in the same order as for non-macro paths - items in modules, extern prelude, tool prelude (see later), standard library prelude, language prelude, but with some extra shadowing restrictions (names from globs and macro expansions cannot shadow names from outer scopes). See the comment in `fn resolve_lexical_macro_path_segment` for more details.
"Tool prelude" currently contains two "tool modules" `rustfmt` and `clippy`, and is searched immediately after extern prelude.
This makes the [possible long-term solution](https://github.com/rust-lang/rfcs/blob/master/text/2103-tool-attributes.md#long-term-solution) for tool attributes exactly equivalent to the existing extern prelude scheme, except that `--extern=my_crate` making crate names available in scope is replaced with something like `--tool=my_tool` making tool names available in scope.
The `tool_attributes` feature is still unstable and `#![feature(tool_attributes)]` now implicitly enables `#![feature(use_extern_macros)]`. `use_extern_macros` is a prerequisite for `tool_attributes`, so their stabilization will happen in the same order.
If `use_extern_macros` is not enabled, then tool attributes are treated as custom attributes (this is temporary, anyway).
Fixes https://github.com/rust-lang/rust/issues/52576
Fixes https://github.com/rust-lang/rust/issues/52512
Fixes https://github.com/rust-lang/rust/issues/51277
cc https://github.com/rust-lang/rust/issues/52269
