There are two interesting kinds of breakage illustrated here:
1. `Box<Trait>` in many contexts is treated as `Box<Trait + 'static>`,
due to [RFC 599]. However, in a type like `&'a Box<Trait>`, the
`Box<Trait>` type will be expanded to `Box<Trait + 'a>`, again due
to [RFC 599]. This, combined with the fix to Issue 25199, leads to
a borrowck problem due the combination of this function signature
(in src/libstd/net/parser.rs):
```rust
fn read_or<T>(&mut self, parsers: &mut [Box<FnMut(&mut Parser) -> Option<T>>]) -> Option<T>;
```
with this call site (again in src/libstd/net/parser.rs):
```rust
fn read_ip_addr(&mut self) -> Option<IpAddr> {
let ipv4_addr = |p: &mut Parser| p.read_ipv4_addr().map(|v4| IpAddr::V4(v4));
let ipv6_addr = |p: &mut Parser| p.read_ipv6_addr().map(|v6| IpAddr::V6(v6));
self.read_or(&mut [Box::new(ipv4_addr), Box::new(ipv6_addr)])
}
```
yielding borrowck errors like:
```
parser.rs:265:27: 265:69 error: borrowed value does not live long enough
parser.rs:265 self.read_or(&mut [Box::new(ipv4_addr), Box::new(ipv6_addr)])
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
(full log at: https://gist.github.com/pnkfelix/e2e80f1a71580f5d3103 )
The issue here is perhaps subtle: the `parsers` argument is
inferred to be taking a slice of boxed objects with the implicit
lifetime bound attached to the `self` parameter to `read_or`.
Meanwhile, the fix to Issue 25199 (added in a forth-coming commit)
is forcing us to assume that each boxed object may have a
destructor that could refer to state of that lifetime, and
*therefore* that inferred lifetime is required to outlive the boxed
object itself.
In this case, the relevant boxed object here is not going to make
any such references; I believe it is just an artifact of how the
expression was built that it is not assigned type:
`Box<FnMut(&mut Parser) -> Option<T> + 'static>`.
(i.e., mucking with the expression is probably one way to fix this
problem).
But the other way to fix it, adopted here, is to change the
`read_or` method type to force make the (presumably-intended)
`'static` bound explicit on the boxed `FnMut` object.
(Note: this is still just the *first* example of breakage.)
2. In `macro_rules.rs`, the `TTMacroExpander` trait defines a method
with signature:
```rust
fn expand<'cx>(&self, cx: &'cx mut ExtCtxt, ...) -> Box<MacResult+'cx>;
```
taking a `&'cx mut ExtCtxt` as an argument and returning a
`Box<MacResult'cx>`.
The fix to Issue 25199 (added in aforementioned forth-coming
commit) assumes that a value of type `Box<MacResult+'cx>` may, in
its destructor, refer to a reference of lifetime `'cx`; thus the
`'cx` lifetime is forced to outlive the returned value.
Meanwhile, within `expand.rs`, the old code was doing:
```rust
match expander.expand(fld.cx, ...).make_pat() { ... => immutable borrow of fld.cx ... }
```
The problem is that the `'cx` lifetime, inferred for the
`expander.expand` call, has now been extended so that it has to
outlive the temporary R-value returned by `expanded.expand`. But
call is also reborrowing `fld.cx` *mutably*, which means that this
reborrow must end before any immutable borrow of `fld.cx`; but
there is one of those within the match body. (Note that the
temporary R-values for the input expression to `match` all live as
long as the whole `match` expression itself (see Issue #3511 and PR
#11585).
To address this, I moved the construction of the pat value into its
own `let`-statement, so that the `Box<MacResult>` will only live
for as long as the initializing expression for the `let`-statement,
and thus allow the subsequent immutable borrow within the `match`.
[RFC 599]: https://github.com/rust-lang/rfcs/blob/master/text/0599-default-object-bound.md
This changes the `ToTokens` implementations for expressions, statements,
etc. with almost-trivial ones that produce `Interpolated(*Nt(...))`
pseudo-tokens. In this way, quasiquote now works the same way as macros
do: already-parsed AST fragments are used as-is, not reparsed.
The `ToSource` trait is removed. Quasiquote no longer involves
pretty-printing at all, which removes the need for the
`encode_with_hygiene` hack. All associated machinery is removed.
A new `Nonterminal` is added, NtArm, which the parser now interpolates.
This is just for quasiquote, not macros (although it could be in the
future).
`ToTokens` is no longer implemented for `Arg` (although this could be
added again) and `Generics` (which I don't think makes sense).
This breaks any compiler extensions that relied on the ability of
`ToTokens` to turn AST fragments back into inspectable token trees. For
this reason, this closes#16987.
As such, this is a [breaking-change].
Fixes#16472.
Fixes#15962.
Fixes#17397.
Fixes#16617.
Changes the style guidelines regarding unit tests to recommend using a
sub-module named "tests" instead of "test" for unit tests as "test"
might clash with imports of libtest.
This is a little bit tricky, since with include_str!, we know that we
are including utf-8 content, so it's safe to store the source as a
String in a FileMap. We don't know that for include_bytes!, but I don't
think we actually need to track the contents anyways, so I'm passing "".
new_filemap does check for the zero length content, and it should be
reasonable, howeven I'm not sure if it would be better to pass None
instead of Some(Rc::new("")) as the src component of a FileMap.
Fixes bug #24348
This is a little bit tricky, since with include_str!, we know that we
are including utf-8 content, so it's safe to store the source as a
String in a FileMap. We don't know that for include_bytes!, but I don't
think we actually need to track the contents anyways, so I'm passing "".
new_filemap does check for the zero length content, and it should be
reasonable, howeven I'm not sure if it would be better to pass None
instead of Some(Rc::new("")) as the src component of a FileMap.
Fixes bug #24348
There are syntax extensions that call `std::rt::begin_unwind` passing it a `usize`. I updated the syntax extension to instead pass `u32`, but for bootstrapping reasons, I needed to create a `#[cfg(stage0)]` version of `std::rt::begin_unwind` and therefore also `panic!`.
`format_args!` uses `#[allow_internal_unstable]` to access internal
functions and structs that are marked unstable. For this to work, the
spans on AST nodes referencing unstable internals must be equal (same
lo/hi values) to the `format_args!` call site, so that the stability
checker can recognize that the AST node was generated by the macro.
* In noop_fold_expr, call new_span in these cases:
- ExprMethodCall's identifier
- ExprField's identifier
- ExprTupField's integer
Calling new_span for ExprMethodCall's identifier is necessary to print
an acceptable diagnostic for write!(&2, ""). We see this error:
<std macros>:2:20: 2:66 error: type `&mut _` does not implement any method in scope named `write_fmt`
<std macros>:2 ( & mut * $ dst ) . write_fmt ( format_args ! ( $ ( $ arg ) * ) ) )
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
With this change, we also see a macro expansion backtrace leading to
the write!(&2, "") call site.
* After fully expanding a macro, we replace the expansion expression's
span with the original span. Call fld.new_span to add a backtrace to
this span. (Note that I'm call new_span after bt.pop(), so the macro
just expanded isn't on the backtrace.)
The motivating example for this change is println!("{}"). The format
string literal is concat!($fmt, "arg") and is inside the libstd macro.
We need to see the backtrace to find the println! call site.
* Add a backtrace to the format_args! format expression span.
Addresses #23459
Statement macros are now treated somewhat like item macros, in that a statement macro can now expand into a series of statements, rather than just a single statement.
This allows statement macros to be nested inside other kinds of macros and expand properly, where previously the expansion would only work when no nesting was present.
See:
- `src/test/run-pass/macro-stmt_macro_in_expr_macro.rs`
- `src/test/run-pass/macro-nested_stmt_macro.rs`
This changes the interface of the MacResult trait. make_stmt has become make_stmts and now returns a vector, rather than a single item. Plugin writers who were implementing MacResult will have breakage, as well as anyone using MacEager::stmt.
See:
- `src/libsyntax/ext/base.rs`
This also causes a minor difference in behavior to the diagnostics produced by certain malformed macros.
See:
- `src/test/compile-fail/macro-incomplete-parse.rs`
Implements pop() on SmallVector, and uses it to expand the final semicolon
in a statement macro expansion more efficiently.
Corrects the placement of the call to fld.cx.bt_pop(). It must run
unconditionally to reverse the corresponding push.
Statement macros are now treated somewhat like item macros, in that a
statement macro can now expand into a series of statements, rather than
just a single statement.
This allows statement macros to be nested inside other kinds of macros and
expand properly, where previously the expansion would only work when no
nesting was present.
See: src/test/run-pass/macro-stmt_macro_in_expr_macro.rs
src/test/run-pass/macro-nested_stmt_macro.rs
This changes the interface of the MacResult trait. make_stmt has become
make_stmts and now returns a vector, rather than a single item. Plugin
writers who were implementing MacResult will have breakage, as well as
anyone using MacEager::stmt.
See: src/libsyntax/ext/base.rs
This also causes a minor difference in behavior to the diagnostics
produced by certain malformed macros.
See: src/test/compile-fail/macro-incomplete-parse.rs
- Functions in parser.rs return PResult<> rather than panicing
- Other functions in libsyntax call panic! explicitly for now if they rely on panicing behaviour.
- 'panictry!' macro added as scaffolding while converting panicing functions.
(This does the same as 'unwrap()' but is easier to grep for and turn into try!())
- Leaves panicing wrappers for the following functions so that the
quote_* macros behave the same:
- parse_expr, parse_item, parse_pat, parse_arm, parse_ty, parse_stmt
RFC pending, but this is the patch that does it.
Totally untested. Likely needs some removed imports. std::collections docs should also be updated to provide better examples.
Closes#23508
