This shrinks:
- ThinTokenStream: 16 to 8 bytes
- TokenTree: 32 to 24 bytes
- TokenStream: 40 to 32 bytes
The only downside is that in a couple of places this requires using
`to_vec()` (which allocates) instead of `sub_slice()`. But those places
are rarely executed, so it doesn't hurt perf.
Overall, this reduces instruction counts on numerous benchmarks by up to
3%.
The syntax crate contains those things concerned purely with syntax
– that is, the AST ("abstract syntax tree"), parser, pretty-printer,
lexer, macro expander, and utilities for traversing ASTs.
For more information about how these things work in rustc, see the
rustc guide:
67f8fb5960