proc_macro: Add an expand_expr method to TokenStream
This feature is aimed at giving proc macros access to powers similar to those used by builtin macros such as `format_args!` or `concat!`. These macros are able to accept macros in place of string literal parameters, such as the format string, as they perform recursive macro expansion while being expanded.
This can be especially useful in many cases thanks to helper macros like `concat!`, `stringify!` and `include_str!` which are often used to construct string literals at compile-time in user code.
For now, this method only allows expanding macros which produce literals, although more expressions will be supported before the method is stabilized.
In earlier versions of this PR, this method exclusively returned `Literal`, and spans on returned literals were stripped of expansion context before being returned to be as conservative as possible about permission leakage. The method's naming has been generalized to eventually support arbitrary expressions, and the context stripping has been removed (https://github.com/rust-lang/rust/pull/87264#discussion_r674863279), which should allow for more general APIs like "format_args_implicits" (https://github.com/rust-lang/rust/issues/67984) to be supported as well.
## API Surface
```rust
impl TokenStream {
pub fn expand_expr(&self) -> Result<TokenStream, ExpandError>;
}
#[non_exhaustive]
pub struct ExpandError;
impl Debug for ExpandError { ... }
impl Display for ExpandError { ... }
impl Error for ExpandError {}
impl !Send for ExpandError {}
impl !Sync for ExpandError {}
```
This feature is aimed at giving proc macros access to powers similar to
those used by builtin macros such as `format_args!` or `concat!`. These
macros are able to accept macros in place of string literal parameters,
such as the format string, as they perform recursive macro expansion
while being expanded.
This can be especially useful in many cases thanks to helper macros like
`concat!`, `stringify!` and `include_str!` which are often used to
construct string literals at compile-time in user code.
For now, this method only allows expanding macros which produce
literals, although more expresisons will be supported before the method
is stabilized.
Append .0 to unsuffixed float if it would otherwise become int token
Previously the unsuffixed f32/f64 constructors of `proc_macro::Literal` would create literal tokens that are definitely not a float:
```rust
Literal::f32_unsuffixed(10.0) // 10
Literal::f32_suffixed(10.0) // 10f32
Literal::f64_unsuffixed(10.0) // 10
Literal::f64_suffixed(10.0) // 10f64
```
Notice that the `10` are actually integer tokens if you were to reparse them, not float tokens.
This diff updates `Literal::f32_unsuffixed` and `Literal::f64_unsuffixed` to produce tokens that unambiguously parse as a float. This matches longstanding behavior of the proc-macro2 crate's implementation of these APIs dating back at least 3.5 years, so it's likely an unobjectionable behavior.
```rust
Literal::f32_unsuffixed(10.0) // 10.0
Literal::f32_suffixed(10.0) // 10f32
Literal::f64_unsuffixed(10.0) // 10.0
Literal::f64_suffixed(10.0) // 10f64
```
Fixes https://github.com/dtolnay/syn/issues/1085.
Improve and test cross-crate hygiene
- Decode the parent expansion for traits and enums in `rustc_resolve`, this was already being used for resolution in typeck
- Avoid suggesting importing names with def-site hygiene, since it's often not useful
- Add more tests
r? `@petrochenkov`
Emit description of the ambiguity as a note.
Co-authored-by: Noah Lev <camelidcamel@gmail.com>
Co-authored-by: Vadim Petrochenkov <vadim.petrochenkov@gmail.com>
The affected crates have had plenty of time to update.
By keeping these as lints rather than making them hard errors,
we ensure that downstream crates will still be able to compile,
even if they transitive depend on broken versions of the affected
crates.
This should hopefully discourage anyone from writing any
new code which relies on the backwards-compatibility behavior.
When `cargo report future-incompatibilities` is stabilized
(see #71249), this will cause dependencies that trigger
this lint to be included in the report.
It was previously cached for modules loaded from `fn get_module`, but not for modules loaded from `fn build_reduced_graph_for_external_crate_res`.
This also makes all foreign modules use their real parent, span and expansion instead of possibly a parent/span/expansion of their reexport.
An ICE happening on attempt to decode expansions for foreign enums and traits is avoided.
Also local enums and traits are now added to the module map.
Use smaller spans for some structured suggestions
Use more accurate suggestion spans for
* argument parse error
* fully qualified path
* missing code block type
* numeric casts
* On suggestions that include deletions, use a diff inspired output format
* When suggesting addition, use `+` as underline
* Color highlight modified span
Support negative numbers in Literal::from_str
proc_macro::Literal has allowed negative numbers in a single literal token ever since Rust 1.29, using https://doc.rust-lang.org/stable/proc_macro/struct.Literal.html#method.isize_unsuffixed and similar constructors.
```rust
let lit = proc_macro::Literal::isize_unsuffixed(-10);
```
However, the suite of constructors on Literal is not sufficient for all use cases, for example arbitrary precision floats, or custom suffixes in FFI macros.
```rust
let lit = proc_macro::Literal::f64_unsuffixed(0.101001000100001000001000000100000001); // :(
let lit = proc_macro::Literal::i???_suffixed(10ulong); // :(
```
For those, macros construct the literal using from_str instead, which preserves arbitrary precision, custom suffixes, base, and digit grouping.
```rust
let lit = "0.101001000100001000001000000100000001".parse::<Literal>().unwrap();
let lit = "10ulong".parse::<Literal>().unwrap();
let lit = "0b1000_0100_0010_0001".parse::<Literal>().unwrap();
```
However, until this PR it was not possible to construct a literal token that is **both** negative **and** preserving of arbitrary precision etc.
This PR fixes `Literal::from_str` to recognize negative integer and float literals.
Various diagnostics clean ups/tweaks
* Always point at macros, including derive macros
* Point at non-local items that introduce a trait requirement
* On private associated item, point at definition
* Always point at macros, including derive macros
* Point at non-local items that introduce a trait requirement
* On private associated item, point at definition
Compute a better `lint_node_id` during expansion
When we need to emit a lint at a macro invocation, we currently use the
`NodeId` of its parent definition (e.g. the enclosing function). This
means that any `#[allow]` / `#[deny]` attributes placed 'closer' to the
macro (e.g. on an enclosing block or statement) will have no effect.
This commit computes a better `lint_node_id` in `InvocationCollector`.
When we visit/flat_map an AST node, we assign it a `NodeId` (earlier
than we normally would), and store than `NodeId` in current
`ExpansionData`. When we collect a macro invocation, the current
`lint_node_id` gets cloned along with our `ExpansionData`, allowing it
to be used if we need to emit a lint later on.
This improves the handling of `#[allow]` / `#[deny]` for
`SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` and some `asm!`-related lints.
The 'legacy derive helpers' lint retains its current behavior
(I've inlined the now-removed `lint_node_id` function), since
there isn't an `ExpansionData` readily available.
