proc_macro/bridge: use the cross-thread executor for nested proc-macros

While working on some other changes in the bridge, I noticed that when
running a nested proc-macro (which is currently only possible using
the unstable `TokenStream::expand_expr`), any symbols held by the
proc-macro client would be invalidated, as the same thread would be used
for the nested macro by default, and the interner doesn't handle nested
use.

After discussing with @eddyb, we decided the best approach might be to
force the use of the cross-thread executor for nested invocations, as it
will never re-use thread-local storage, avoiding the issue. This
shouldn't impact performance, as expand_expr is still unstable, and
infrequently used.

This was chosen rather than making the client symbol interner handle
nested invocations, as that would require replacing the internal
interner `Vec` with a `BTreeMap` (as valid symbol id ranges could now be
disjoint), and the symbol interner is known to be fairly perf-sensitive.

This patch adds checks to the execution strategy to use the cross-thread
executor when doing nested invocations. An alternative implementation
strategy could be to track this information in the `ExtCtxt`, however a
thread-local in the `proc_macro` crate was chosen to add an assertion so
that `rust-analyzer` is aware of the issue if it implements
`expand_expr` in the future.

r? @eddyb
This commit is contained in:
Nika Layzell 2022-09-04 12:53:29 -04:00
parent b11bf65e4a
commit efda49712b
2 changed files with 56 additions and 4 deletions

View file

@ -80,13 +80,21 @@ fn assert_ts_eq(lhs: &TokenStream, rhs: &TokenStream) {
pub fn expand_expr_is(input: TokenStream) -> TokenStream {
let mut iter = input.into_iter();
let mut expected_tts = Vec::new();
loop {
let comma = loop {
match iter.next() {
Some(TokenTree::Punct(ref p)) if p.as_char() == ',' => break,
Some(TokenTree::Punct(p)) if p.as_char() == ',' => break p,
Some(tt) => expected_tts.push(tt),
None => panic!("expected comma"),
}
}
};
// Make sure that `Ident` and `Literal` objects from this proc-macro's
// environment are not invalidated when `expand_expr` recursively invokes
// another macro by taking a local copy, and checking it after the fact.
let pre_expand_span = comma.span();
let pre_expand_ident = Ident::new("ident", comma.span());
let pre_expand_literal = Literal::string("literal");
let pre_expand_call_site = Span::call_site();
let expected = expected_tts.into_iter().collect::<TokenStream>();
let expanded = iter.collect::<TokenStream>().expand_expr().expect("expand_expr failed");
@ -100,6 +108,15 @@ pub fn expand_expr_is(input: TokenStream) -> TokenStream {
// Also compare the raw tts to make sure they line up.
assert_ts_eq(&expected, &expanded);
assert!(comma.span().eq(&pre_expand_span), "pre-expansion span is still equal");
assert_eq!(pre_expand_ident.to_string(), "ident", "pre-expansion identifier is still valid");
assert_eq!(
pre_expand_literal.to_string(),
"\"literal\"",
"pre-expansion literal is still valid"
);
assert!(Span::call_site().eq(&pre_expand_call_site), "pre-expansion call-site is still equal");
TokenStream::new()
}