This commit is a reorganization of the `proc_macro` crate's public user-facing
API. This is the result of a number of discussions at the recent Rust All-Hands
where we're hoping to get the `proc_macro` crate into ship shape for
stabilization of a subset of its functionality in the Rust 2018 release.
The reorganization here is motivated by experiences from the `proc-macro2`,
`quote`, and `syn` crates on crates.io (and other crates which depend on them).
The main focus is future flexibility along with making a few more operations
consistent and/or fixing bugs. A summary of the changes made from today's
`proc_macro` API is:
* The `TokenNode` enum has been removed and the public fields of `TokenTree`
have also been removed. Instead the `TokenTree` type is now a public enum
(what `TokenNode` was) and each variant is an opaque struct which internally
contains `Span` information. This makes the various tokens a bit more
consistent, require fewer wrappers, and otherwise provides good
future-compatibility as opaque structs are easy to modify later on.
* `Literal` integer constructors have been expanded to be unambiguous as to what
they're doing and also allow for more future flexibility. Previously
constructors like `Literal::float` and `Literal::integer` were used to create
unsuffixed literals and the concrete methods like `Literal::i32` would create
a suffixed token. This wasn't immediately clear to all users (the
suffixed/unsuffixed aspect) and having *one* constructor for unsuffixed
literals required us to pick a largest type which may not always be true. To
fix these issues all constructors are now of the form
`Literal::i32_unsuffixed` or `Literal::i32_suffixed` (for all integral types).
This should allow future compatibility as well as being immediately clear
what's suffixed and what isn't.
* Each variant of `TokenTree` internally contains a `Span` which can also be
configured via `set_span`. For example `Literal` and `Term` now both
internally contain a `Span` rather than having it stored in an auxiliary
location.
* Constructors of all tokens are called `new` now (aka `Term::intern` is gone)
and most do not take spans. Manufactured tokens typically don't have a fresh
span to go with them and the span is purely used for error-reporting
**except** the span for `Term`, which currently affects hygiene. The default
spans for all these constructed tokens is `Span::call_site()` for now.
The `Term` type's constructor explicitly requires passing in a `Span` to
provide future-proofing against possible hygiene changes. It's intended that a
first pass of stabilization will likely only stabilize `Span::call_site()`
which is an explicit opt-in for "I would like no hygiene here please". The
intention here is to make this explicit in procedural macros to be
forwards-compatible with a hygiene-specifying solution.
* Some of the conversions for `TokenStream` have been simplified a little.
* The `TokenTreeIter` iterator was renamed to `token_stream::IntoIter`.
Overall the hope is that this is the "final pass" at the API of `TokenStream`
and most of `TokenTree` before stabilization. Explicitly left out here is any
changes to `Span`'s API which will likely need to be re-evaluated before
stabilization.
All changes in this PR have already been reflected to the [`proc-macro2`],
`quote`, and `syn` crates. New versions of all these crates have also been
published to crates.io.
Once this lands in nightly I plan on making an internals post again summarizing
the changes made here and also calling on all macro authors to give the APIs a
spin and see how they work. Hopefully pending no major issues we can then have
an FCP to stabilize later this cycle!
[`proc-macro2`]: https://docs.rs/proc-macro2/0.3.1/proc_macro2/
Don't include bang in macro replacement suggestion
When we suggest the replacement for a macro we include the "!" in the suggested replacement but the span only contains the name of the macro itself. Using that replacement would cause a duplicate "!" in the resulting code.
I originally tried to extend the span to be replaced by 1 byte in rust-lang/rust#47424. However, @zackmdavis pointed out that there can be whitespace between the macro name and the bang.
Instead, just remove the bang from the suggested replacement.
Fixes#47418
r? @estebank
When we suggest the replacement for a macro we include the "!" in the
suggested replacement but the span only contains the name of the macro
itself. Using that replacement would cause a duplicate "!" in the
resulting code.
I originally tried to extend the span to be replaced by 1 byte in
rust-lang/rust#47424. However, @zackmdavis pointed out that there can be
whitespace between the macro name and the bang.
Instead, just remove the bang from the suggested replacement.
Fixes#47418
When a ui-fulldeps comparison fails it suggests running
update-references.sh:
```
src/test/ui-fulldeps/update-references.sh 'rust/build/x86_64-apple-darwin/test/ui-fulldeps' 'resolve-error.rs'
```
This does not work as update-references.sh isn't executable. The other
update-references.sh in the ui directory is already executable so this looks
like an oversight.
It tested #44953.
`log` macros in newer versions are no longer recursive, so these duplicated
error messages (about unstable feature uses) previously occurring at each
level of recursion are no longer possible, even with the fix by #45540.
Furthermore this test breaks when multiple versions of `log` are in the
sysroot (`log 0.3.9` depends on`log 0.4.1`)
This commit introduces the ability to create and emit `Diagnostic`
structures from proc-macros, allowing for proc-macro authors to emit
warning, error, note, and help messages just like the compiler does.
In preparation for incremental compilation this commit refactors the lint
handling infrastructure in the compiler to be more "eager" and overall more
incremental-friendly. Many passes of the compiler can emit lints at various
points but before this commit all lints were buffered in a table to be emitted
at the very end of compilation. This commit changes these lints to be emitted
immediately during compilation using pre-calculated lint level-related data
structures.
Linting today is split into two phases, one set of "early" lints run on the
`syntax::ast` and a "late" set of lints run on the HIR. This commit moves the
"early" lints to running as late as possible in compilation, just before HIR
lowering. This notably means that we're catching resolve-related lints just
before HIR lowering. The early linting remains a pass very similar to how it was
before, maintaining context of the current lint level as it walks the tree.
Post-HIR, however, linting is structured as a method on the `TyCtxt` which
transitively executes a query to calculate lint levels. Each request to lint on
a `TyCtxt` will query the entire crate's 'lint level data structure' and then go
from there about whether the lint should be emitted or not.
The query depends on the entire HIR crate but should be very quick to calculate
(just a quick walk of the HIR) and the red-green system should notice that the
lint level data structure rarely changes, and should hopefully preserve
incrementality.
Overall this resulted in a pretty big change to the test suite now that lints
are emitted much earlier in compilation (on-demand vs only at the end). This in
turn necessitated the addition of many `#![allow(warnings)]` directives
throughout the compile-fail test suite and a number of updates to the UI test
suite.
The 'run-pass' header cause a 'ui' test to execute the result. It is used
to test the lint output, at the same time ensure those lints won't cause
the source code to become compile-fail.
12 run-pass/run-pass-fulldeps tests gained the header and are moved to
ui/ui-fulldeps. After this move, no run-pass/run-pass-fulldeps tests should
rely on the compiler's JSON message. This allows us to stop passing
`--error-format json` in run-pass tests, thus fixing #36516.
