user0/rust
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
rust/src
History
bors 4d526e0d14 Auto merge of #40939 - jseyfried:proc_macro_api, r=nrc 
proc_macro: implement `TokenTree`, `TokenKind`, hygienic `quote!`, and other API

All new API is gated behind `#![feature(proc_macro)]` and may be used with `#[proc_macro]`, `#[proc_macro_attribute]`, and `#[proc_macro_derive]` procedural macros.

More specifically, this PR adds the following in `proc_macro`:
```rust
// `TokenStream` constructors:
impl TokenStream { fn empty() -> TokenStream { ... } }
impl From<TokenTree> for TokenStream { ... }
impl From<TokenKind> for TokenStream { ... }
impl<T: Into<TokenStream>> FromIterator<T> for TokenStream { ... }
macro quote($($t:tt)*) { ... } // A hygienic `TokenStream` quoter

// `TokenStream` destructuring:
impl TokenStream { fn is_empty(&self) -> bool { ... } }
impl IntoIterator for TokenStream { type Item = TokenTree; ... }

struct TokenTree { span: Span, kind: TokenKind }
impl From<TokenKind> for TokenTree { ... }
impl Display for TokenTree { ... }

struct Span { ... } // a region of source code along with expansion/hygiene information
impl Default for Span { ... } // a span from the current procedural macro definition
impl Span { fn call_site() -> Span { ... } } // the call site of the current expansion
fn quote_span(span: Span) -> TokenStream;

enum TokenKind {
    Group(Delimiter, TokenStream), // A delimited sequence, e.g. `( ... )`
    Term(Term), // a unicode identifier, lifetime ('a), or underscore
    Op(char, Spacing), // a punctuation character (`+`, `,`, `$`, etc.).
    Literal(Literal), // a literal character (`'a'`), string (`"hello"`), or number (`2.3`)
}

enum Delimiter {
    Parenthesis, // `( ... )`
    Brace, // `[ ... ]`
    Bracket, // `{ ... }`
    None, // an implicit delimiter, e.g. `$var`, where $var is  `...`.
}

struct Term { ... } // An interned string
impl Term {
    fn intern(string: &str) -> Symbol { ... }
    fn as_str(&self) -> &str { ... }
}

enum Spacing {
    Alone, // not immediately followed by another `Op`, e.g. `+` in `+ =`.
    Joint, // immediately followed by another `Op`, e.g. `+` in `+=`
}

struct Literal { ... }
impl Display for Literal { ... }
impl Literal {
    fn integer(n: i128) -> Literal { .. } // unsuffixed integer literal
    fn float(n: f64) -> Literal { .. } // unsuffixed floating point literal
    fn u8(n: u8) -> Literal { ... } // similarly: i8, u16, i16, u32, i32, u64, i64, f32, f64
    fn string(string: &str) -> Literal { ... }
    fn character(ch: char) -> Literal { ... }
    fn byte_string(bytes: &[u8]) -> Literal { ... }
}
```
For details on `quote!` hygiene, see [this example](https://github.com/rust-lang/rust/pull/40939/commits/20a90485c040df87a667e9b6ee38e4d8a7d7fc5d) and [declarative macros 2.0](https://github.com/rust-lang/rust/pull/40847).

r? @nrc
2017-07-05 21:16:34 +00:00
..
bootstrap rustbuild: Only -Zsave-analysis for libstd 2017-07-05 13:51:34 -07:00
build_helper rustbuild: Add ./x.py test --no-fail-fast 2017-06-02 09:27:44 -07:00
ci Restore old emscripten.sh for use by asmjs 2017-06-24 11:35:48 -07:00
compiler-rt@c8a8767c56 support default impl for specialization 2017-04-25 05:28:22 +00:00
doc Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
driver
etc Merge crate collections into alloc 2017-06-13 23:37:34 -07:00
grammar changed upper bound digit in octal rule to 7 2017-06-13 17:08:12 -03:00
jemalloc@11bfb0dcf8 Revert "Update jemalloc to 4.5.0" 2017-06-09 20:23:38 +02:00
liballoc Auto merge of #43050 - stjepang:doc-vec-fix-parens, r=frewsxcv 2017-07-05 02:04:07 +00:00
liballoc_jemalloc Stop disabling fill in jemalloc 2017-06-25 10:58:12 -07:00
liballoc_system Improve reallocation in alloc_system on Windows 2017-06-02 06:29:58 -04:00
libarena Bump version and stage0 compiler 2017-06-19 22:25:05 -07:00
libbacktrace
libcollections Revert "Stabilize RangeArgument" 2017-06-30 08:34:53 -10:00
libcompiler_builtins Support VS 2017 2017-06-01 20:41:38 +00:00
libcore Rollup merge of #43043 - sfackler:reverse-stability, r=Mark-Simulacrum 2017-07-04 07:41:44 -06:00
libfmt_macros Bump version and stage0 compiler 2017-06-19 22:25:05 -07:00
libgetopts Switch to the crates.io getopts crate 2017-06-20 12:43:12 -07:00
libgraphviz Removed as many "```ignore" as possible. 2017-06-23 15:31:53 +08:00
liblibc@2015cf17a6 Update libc to 0.2.24 2017-06-20 13:42:52 +02:00
libpanic_abort
libpanic_unwind std: Avoid panics in rust_eh_personality 2017-06-08 07:06:43 -07:00
libproc_macro Address review comments. 2017-06-26 02:06:34 +00:00
libprofiler_builtins Fix gcc version required by libprofiler_builtins 2017-06-13 20:21:58 +01:00
librand Delete deprecated & unstable range-specific step_by 2017-07-01 19:18:02 -07:00
librustc Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
librustc_asan rustc_{a,l,m,t}san: re-run build script if env var LLVM_CONFIG changes 2017-06-30 17:35:00 +00:00
librustc_back Auto merge of #42784 - tlively:wasm-bot, r=alexcrichton 2017-06-24 22:34:08 +00:00
librustc_bitflags Bump version and stage0 compiler 2017-06-19 22:25:05 -07:00
librustc_borrowck Shift mir-dataflow from rustc_borrowck to rustc_mir crate. 2017-06-28 13:59:12 +02:00
librustc_const_eval rustc: move the PolyFnSig out of TyFnDef. 2017-06-27 16:39:52 +03:00
librustc_const_math Bump version and stage0 compiler 2017-06-19 22:25:05 -07:00
librustc_data_structures Revert "Stabilize RangeArgument" 2017-06-30 08:34:53 -10:00
librustc_driver Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
librustc_errors Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
librustc_incremental Remove the remaining feature gates 2017-07-02 21:29:39 +02:00
librustc_lint Rollup merge of #42886 - durka:pplmm-mwe, r=petrochenkov 2017-06-29 08:40:05 +00:00
librustc_llvm Auto merge of #42971 - stepancheg:ir-demangle, r=nagisa 2017-07-01 05:52:08 +00:00
librustc_lsan rustc_{a,l,m,t}san: re-run build script if env var LLVM_CONFIG changes 2017-06-30 17:35:00 +00:00
librustc_metadata Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
librustc_mir Auto merge of #42924 - pnkfelix:mir-dataflow, r=arielb1 2017-06-30 03:56:33 +00:00
librustc_msan rustc_{a,l,m,t}san: re-run build script if env var LLVM_CONFIG changes 2017-06-30 17:35:00 +00:00
librustc_passes report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
librustc_platform_intrinsics Bump version and stage0 compiler 2017-06-19 22:25:05 -07:00
librustc_plugin Removed as many "```ignore" as possible. 2017-06-23 15:31:53 +08:00
librustc_privacy rustc: move the PolyFnSig out of TyFnDef. 2017-06-27 16:39:52 +03:00
librustc_resolve Make $crate a keyword 2017-06-29 15:19:52 +03:00
librustc_save_analysis Rollup merge of #42766 - nrc:versions, r=nagisa 2017-06-21 10:40:17 -04:00
librustc_trans Output line column info when panicking 2017-07-02 13:53:29 +02:00
librustc_tsan rustc_{a,l,m,t}san: re-run build script if env var LLVM_CONFIG changes 2017-06-30 17:35:00 +00:00
librustc_typeck report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
librustdoc Auto merge of #40939 - jseyfried:proc_macro_api, r=nrc 2017-07-05 21:16:34 +00:00
libserialize Removed as many "```ignore" as possible. 2017-06-23 15:31:53 +08:00
libstd Auto merge of #43051 - Mark-Simulacrum:rollup, r=Mark-Simulacrum 2017-07-04 15:58:24 +00:00
libstd_unicode [libstd_unicode] Upgrade to Unicode 10.0.0 2017-06-30 17:25:28 -06:00
libsyntax Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
libsyntax_ext Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
libsyntax_pos Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
libterm Bump version and stage0 compiler 2017-06-19 22:25:05 -07:00
libtest only show allowed failure count if there are allowed failures 2017-06-25 12:23:20 -04:00
libunwind Haiku: fix initial platform support 2017-04-22 13:47:36 +12:00
llvm@8e1b4fedfa Rebase LLVM on top of LLVM 4.0.1 2017-06-27 18:09:52 +03:00
rt Remove the in-tree flate crate 2017-06-20 07:11:29 -07:00
rtstartup Update stage0 bootstrap compiler 2017-04-29 12:11:14 -07:00
rustc
rustllvm Auto merge of #42993 - stepancheg:editorconfig, r=brson 2017-07-04 01:08:15 +00:00
test Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
tools Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
Cargo.lock Merge remote-tracking branch 'origin/master' into proc_macro_api 2017-07-05 08:42:13 -07:00
Cargo.toml Update cargo/rls submodules and dependencies 2017-06-17 12:00:49 -07:00
stage0.txt Bump version and stage0 compiler 2017-06-19 22:25:05 -07:00