src/doc/reference/src/procedural-macros.md

   1 ## Procedural Macros
   2
   3 *Procedural macros* allow creating syntax extensions as execution of a function.
   4 Procedural macros come in one of three flavors:
   5
   6 * [Function-like macros] - `custom!(...)`
   7 * [Derive mode macros] - `#[derive(CustomMode)]`
   8 * [Attribute macros] - `#[CustomAttribute]`
   9
  10 Procedural macros allow you to run code at compile time that operates over Rust
  11 syntax, both consuming and producing Rust syntax. You can sort of think of
  12 procedural macros as functions from an AST to another AST.
  13
  14 Procedural macros must be defined in a crate with the [crate type] of
  15 `proc-macro`.
  16
  17 > **Note**: When using Cargo, Procedural macro crates are defined with the
  18 > `proc-macro` key in your manifest:
  19 >
  20 > ```toml
  21 > [lib]
  22 > proc-macro = true
  23 > ```
  24
  25 As functions, they must either return syntax, panic, or loop endlessly. Returned
  26 syntax either replaces or adds the syntax depending on the kind of procedural
  27 macro. Panics are caught by the compiler and are turned into a compiler error.
  28 Endless loops are not caught by the compiler which hangs the compiler.
  29
  30 Procedural macros run during compilation, and thus have the same resources that
  31 the compiler has. For example, standard input, error, and output are the same
  32 that the compiler has access to. Similarly, file access is the same. Because
  33 of this, procedural macros have the same security concerns that [Cargo's
  34 build scripts] have.
  35
  36 Procedural macros have two ways of reporting errors. The first is to panic. The
  37 second is to emit a [`compile_error`] macro invocation.
  38
  39 ### The `proc_macro` crate
  40
  41 Procedural macro crates almost always will link to the compiler-provided
  42 [`proc_macro` crate]. The `proc_macro` crate provides types required for
  43 writing procedural macros and facilities to make it easier.
  44
  45 This crate primarily contains a [`TokenStream`] type. Procedural macros operate
  46 over *token streams* instead of AST nodes, which is a far more stable interface
  47 over time for both the compiler and for procedural macros to target. A
  48 *token stream* is roughly equivalent to `Vec<TokenTree>` where a `TokenTree`
  49 can roughly be thought of as lexical token. For example `foo` is an `Ident`
  50 token, `.` is a `Punct` token, and `1.2` is a `Literal` token. The `TokenStream`
  51 type, unlike `Vec<TokenTree>`, is cheap to clone.
  52
  53 All tokens have an associated `Span`. A `Span` is an opaque value that cannot
  54 be modified but can be manufactured. `Span`s represent an extent of source
  55 code within a program and are primarily used for error reporting. You can modify
  56 the `Span` of any token.
  57
  58 ### Procedural macro hygiene
  59
  60 Procedural macros are *unhygienic*. This means they behave as if the output
  61 token stream was simply written inline to the code it's next to. This means that
  62 it's affected by external items and also affects external imports.
  63
  64 Macro authors need to be careful to ensure their macros work in as many contexts
  65 as possible given this limitation. This often includes using absolute paths to
  66 items in libraries (for example, `::std::option::Option` instead of `Option`) or
  67 by ensuring that generated functions have names that are unlikely to clash with
  68 other functions (like `__internal_foo` instead of `foo`).
  69
  70 ### Function-like procedural macros
  71
  72 *Function-like procedural macros* are procedural macros that are invoked using
  73 the macro invocation operator (`!`).
  74
  75 These macros are defined by a [public]&#32;[function] with the `proc_macro`
  76 [attribute] and a signature of `(TokenStream) -> TokenStream`. The input
  77 [`TokenStream`] is what is inside the delimiters of the macro invocation and the
  78 output [`TokenStream`] replaces the entire macro invocation. It may contain an
  79 arbitrary number of [items]. These macros cannot expand to syntax that defines
  80 new `macro_rule` style macros.
  81
  82 For example, the following macro definition ignores its input and outputs a
  83 function `answer` into its scope.
  84
  85 ```rust,ignore
  86 extern crate proc_macro;
  87 use proc_macro::TokenStream;
  88
  89 #[proc_macro]
  90 pub fn make_answer(_item: TokenStream) -> TokenStream {
  91     "fn answer() -> u32 { 42 }".parse().unwrap()
  92 }
  93 ```
  94
  95 And then we use it a binary crate to print "42" to standard output.
  96
  97 ```rust,ignore
  98 extern crate proc_macro_examples;
  99 use proc_macro_examples::make_answer;
 100
 101 make_answer!();
 102
 103 fn main() {
 104     println!("{}", answer());
 105 }
 106 ```
 107
 108 These macros are only invokable in [modules]. They cannot even be invoked to
 109 create [item declaration statements]. Furthermore, they must either be invoked
 110 with curly braces and no semicolon or a different delimiter followed by a
 111 semicolon. For example, `make_answer` from the previous example can be invoked
 112 as `make_answer!{}`, `make_answer!();` or `make_answer![];`.
 113
 114 ### Derive mode macros
 115
 116 *Derive mode macros* define new modes for the `derive` [attribute]. These macros
 117 define new [items] given the token stream of a [struct], [enum], or [union].
 118 They also define [derive mode helper attributes].
 119
 120 Custom deriver modes are defined by a [public]&#32;[function] with the
 121 `proc_macro_derive` attribute and a signature of `(TokenStream) -> TokenStream`.
 122
 123 The input [`TokenStream`] is the token stream of the item that has the `derive`
 124 attribute on it. The output [`TokenStream`] must be a set of items that are
 125 then appended to the [module] or [block] that the item from the input
 126 [`TokenStream`] is in.
 127
 128 The following is an example of a derive mode macro. Instead of doing anything
 129 useful with its input, it just appends a function `answer`.
 130
 131 ```rust,ignore
 132 extern crate proc_macro;
 133 use proc_macro::TokenStream;
 134
 135 #[proc_macro_derive(AnswerFn)]
 136 pub fn derive_answer_fn(_item: TokenStream) -> TokenStream {
 137     "fn answer() -> u32 { 42 }".parse().unwrap()
 138 }
 139 ```
 140
 141 And then using said derive mode:
 142
 143 ```rust,ignore
 144 extern crate proc_macro_examples;
 145 use proc_macro_examples::AnswerFn;
 146
 147 #[derive(AnswerFn)]
 148 struct Struct;
 149
 150 fn main() {
 151     assert_eq!(42, answer());
 152 }
 153 ```
 154
 155 #### Derive mode helper attributes
 156
 157 Derive mode macros can add additional [attributes] into the scope of the [item]
 158 they are on. Said attributes are called *derive mode helper attributes*. These
 159 attributes are [inert], and their only purpose is to be fed into the derive
 160 mode macro that defined them. That said, they can be seen by all macros.
 161
 162 The way to define helper attributes is to put an `attributes` key in the
 163 `proc_macro_derive` macro with a comma separated list of identifiers that are
 164 the names of the helper attributes.
 165
 166 For example, the following derive mode macro defines a helper attribute
 167 `helper`, but ultimately doesn't do anything with it.
 168
 169 ```rust,ignore
 170 # #[crate_type="proc-macro"]
 171 # extern crate proc_macro;
 172 # use proc_macro::TokenStream;
 173
 174 #[proc_macro_derive(HelperAttr, attributes(helper))]
 175 pub fn derive_helper_attr(_item: TokenStream) -> TokenStream {
 176     TokenStream::new();
 177 }
 178 ```
 179
 180 And then usage on the derive mode on a struct:
 181
 182 ```rust,ignore
 183 # #![crate_type="proc-macro"]
 184 # extern crate proc_macro_examples;
 185 # use proc_macro_examples::HelperAttr;
 186
 187 #[derive(HelperAttr)]
 188 struct Struct {
 189     #[helper] field: ()
 190 }
 191 ```
 192
 193 ### Attribute macros
 194
 195 *Attribute macros* define new [attributes] which can be attached to [items].
 196
 197 Attribute macros are defined by a [public]&#32;[function] with the
 198 `proc_macro_attribute` [attribute] that a signature of
 199 `(TokenStream, TokenStream) -> TokenStream`. The first [`TokenStream`] is the
 200 attribute's metaitems, not including the delimiters. If the attribute is written
 201 without a metaitem, the attribute [`TokenStream`] is empty. The second
 202 [`TokenStream`] is of the rest of the [item] including other [attributes] on the
 203 [item]. The returned [`TokenStream`] replaces the [item] with an arbitrary
 204 number of [items]. These macros cannot expand to syntax that defines new
 205 `macro_rule` style macros.
 206
 207 For example, this attribute macro takes the input stream and returns it as is,
 208 effectively being the no-op of attributes.
 209
 210 ```rust,ignore
 211 # #![crate_type = "proc-macro"]
 212 # extern crate proc_macro;
 213 # use proc_macro::TokenStream;
 214
 215 #[proc_macro_attribute]
 216 pub fn return_as_is(_attr: TokenStream, item: TokenStream) -> TokenStream {
 217     item
 218 }
 219 ```
 220
 221 This following example shows the stringified [`TokenStream`s] that the attribute
 222 macros see. The output will show in the output of the compiler. The output is
 223 shown in the comments after the function prefixed with "out:".
 224
 225 ```rust,ignore
 226 // my-macro/src/lib.rs
 227 # extern crate proc_macro;
 228 # use proc_macro::TokenStream;
 229
 230 #[proc_macro_attribute]
 231 pub fn show_streams(attr: TokenStream, input: TokenStream) -> TokenStream {
 232     println!("attr: \"{}\"", attr.to_string());
 233     println!("item: \"{}\"', input.to_string());
 234     item
 235 }
 236 ```
 237
 238 ```rust,ignore
 239 // src/lib.rs
 240 extern crate my_macro;
 241
 242 use my_macro::show_streams;
 243
 244 // Example: Basic function
 245 #[show_streams]
 246 fn invoke1() {}
 247 // out: attr: ""
 248 // out: item: "fn invoke1() { }"
 249
 250 // Example: Attribute has a metaitem
 251 #[show_streams(bar)]
 252 fn invoke2() {}
 253 // out: attr: "bar"
 254 // out: item: "fn invoke2() {}"
 255
 256 // Example: Multiple words in metaitem
 257 #[show_streams(multiple words)]
 258 fn invoke3() {}
 259 // out: attr: "multiple words"
 260 // out: item: "fn invoke3() {}"
 261
 262 // Example:
 263 #[show_streams { delimiters }]
 264 fn invoke4() {}
 265 // out: "delimiters"
 266 // out: "fn invoke4() {}"
 267 ```
 268
 269 [`TokenStream`]: ../proc_macro/struct.TokenStream.html
 270 [`TokenStream`s]: ../proc_macro/struct.TokenStream.html
 271 [`compile_error`]: ../std/macro.compile_error.html
 272 [`derive`]: attributes.html#derive
 273 [`proc_macro` crate]: ../proc_macro/index.html
 274 [Cargo's build scripts]: ../cargo/reference/build-scripts.html
 275 [Derive mode macros]: #derive-mode-macros
 276 [Attribute macros]: #attribute-macros
 277 [Function-like macros]: #function-like-procedural-macros
 278 [attribute]: attributes.html
 279 [attributes]: attributes.html
 280 [custom attributes]: attributes.html
 281 [crate type]: linkage.html
 282 [derive mode helper attributes]: #derive-mode-helper-attributes
 283 [enum]: items/enumerations.html
 284 [inert]: attributes.html#active-and-inert-attributes
 285 [item]: items.html
 286 [item declaration statements]: statements.html#item-declarations
 287 [items]: items.html
 288 [function]: items/functions.html
 289 [macro]: macros.html
 290 [module]: items/modules.html
 291 [modules]: items/modules.html
 292 [procedural macro tutorial]: ../book/2018-edition/appendix-04-macros.html#procedural-macros-for-custom-derive
 293 [public]: visibility-and-privacy.html
 294 [struct]: items/structs.html
 295 [unions]: items/unions.html