]> git.proxmox.com Git - rustc.git/blob - src/doc/book/macros.md
Imported Upstream version 1.10.0+dfsg1
[rustc.git] / src / doc / book / macros.md
1 % Macros
2
3 By now you’ve learned about many of the tools Rust provides for abstracting and
4 reusing code. These units of code reuse have a rich semantic structure. For
5 example, functions have a type signature, type parameters have trait bounds,
6 and overloaded functions must belong to a particular trait.
7
8 This structure means that Rust’s core abstractions have powerful compile-time
9 correctness checking. But this comes at the price of reduced flexibility. If
10 you visually identify a pattern of repeated code, you may find it’s difficult
11 or cumbersome to express that pattern as a generic function, a trait, or
12 anything else within Rust’s semantics.
13
14 Macros allow us to abstract at a syntactic level. A macro invocation is
15 shorthand for an "expanded" syntactic form. This expansion happens early in
16 compilation, before any static checking. As a result, macros can capture many
17 patterns of code reuse that Rust’s core abstractions cannot.
18
19 The drawback is that macro-based code can be harder to understand, because
20 fewer of the built-in rules apply. Like an ordinary function, a well-behaved
21 macro can be used without understanding its implementation. However, it can be
22 difficult to design a well-behaved macro! Additionally, compiler errors in
23 macro code are harder to interpret, because they describe problems in the
24 expanded code, not the source-level form that developers use.
25
26 These drawbacks make macros something of a "feature of last resort". That’s not
27 to say that macros are bad; they are part of Rust because sometimes they’re
28 needed for truly concise, well-abstracted code. Just keep this tradeoff in
29 mind.
30
31 # Defining a macro
32
33 You may have seen the `vec!` macro, used to initialize a [vector][vector] with
34 any number of elements.
35
36 [vector]: vectors.html
37
38 ```rust
39 let x: Vec<u32> = vec![1, 2, 3];
40 # assert_eq!(x, [1, 2, 3]);
41 ```
42
43 This can’t be an ordinary function, because it takes any number of arguments.
44 But we can imagine it as syntactic shorthand for
45
46 ```rust
47 let x: Vec<u32> = {
48 let mut temp_vec = Vec::new();
49 temp_vec.push(1);
50 temp_vec.push(2);
51 temp_vec.push(3);
52 temp_vec
53 };
54 # assert_eq!(x, [1, 2, 3]);
55 ```
56
57 We can implement this shorthand, using a macro: [^actual]
58
59 [^actual]: The actual definition of `vec!` in libcollections differs from the
60 one presented here, for reasons of efficiency and reusability.
61
62 ```rust
63 macro_rules! vec {
64 ( $( $x:expr ),* ) => {
65 {
66 let mut temp_vec = Vec::new();
67 $(
68 temp_vec.push($x);
69 )*
70 temp_vec
71 }
72 };
73 }
74 # fn main() {
75 # assert_eq!(vec![1,2,3], [1, 2, 3]);
76 # }
77 ```
78
79 Whoa, that’s a lot of new syntax! Let’s break it down.
80
81 ```rust,ignore
82 macro_rules! vec { ... }
83 ```
84
85 This says we’re defining a macro named `vec`, much as `fn vec` would define a
86 function named `vec`. In prose, we informally write a macro’s name with an
87 exclamation point, e.g. `vec!`. The exclamation point is part of the invocation
88 syntax and serves to distinguish a macro from an ordinary function.
89
90 ## Matching
91
92 The macro is defined through a series of rules, which are pattern-matching
93 cases. Above, we had
94
95 ```rust,ignore
96 ( $( $x:expr ),* ) => { ... };
97 ```
98
99 This is like a `match` expression arm, but the matching happens on Rust syntax
100 trees, at compile time. The semicolon is optional on the last (here, only)
101 case. The "pattern" on the left-hand side of `=>` is known as a ‘matcher’.
102 These have [their own little grammar] within the language.
103
104 [their own little grammar]: ../reference.html#macros
105
106 The matcher `$x:expr` will match any Rust expression, binding that syntax tree
107 to the ‘metavariable’ `$x`. The identifier `expr` is a ‘fragment specifier’;
108 the full possibilities are enumerated later in this chapter.
109 Surrounding the matcher with `$(...),*` will match zero or more expressions,
110 separated by commas.
111
112 Aside from the special matcher syntax, any Rust tokens that appear in a matcher
113 must match exactly. For example,
114
115 ```rust,ignore
116 macro_rules! foo {
117 (x => $e:expr) => (println!("mode X: {}", $e));
118 (y => $e:expr) => (println!("mode Y: {}", $e));
119 }
120
121 fn main() {
122 foo!(y => 3);
123 }
124 ```
125
126 will print
127
128 ```text
129 mode Y: 3
130 ```
131
132 With
133
134 ```rust,ignore
135 foo!(z => 3);
136 ```
137
138 we get the compiler error
139
140 ```text
141 error: no rules expected the token `z`
142 ```
143
144 ## Expansion
145
146 The right-hand side of a macro rule is ordinary Rust syntax, for the most part.
147 But we can splice in bits of syntax captured by the matcher. From the original
148 example:
149
150 ```rust,ignore
151 $(
152 temp_vec.push($x);
153 )*
154 ```
155
156 Each matched expression `$x` will produce a single `push` statement in the
157 macro expansion. The repetition in the expansion proceeds in "lockstep" with
158 repetition in the matcher (more on this in a moment).
159
160 Because `$x` was already declared as matching an expression, we don’t repeat
161 `:expr` on the right-hand side. Also, we don’t include a separating comma as
162 part of the repetition operator. Instead, we have a terminating semicolon
163 within the repeated block.
164
165 Another detail: the `vec!` macro has *two* pairs of braces on the right-hand
166 side. They are often combined like so:
167
168 ```rust,ignore
169 macro_rules! foo {
170 () => {{
171 ...
172 }}
173 }
174 ```
175
176 The outer braces are part of the syntax of `macro_rules!`. In fact, you can use
177 `()` or `[]` instead. They simply delimit the right-hand side as a whole.
178
179 The inner braces are part of the expanded syntax. Remember, the `vec!` macro is
180 used in an expression context. To write an expression with multiple statements,
181 including `let`-bindings, we use a block. If your macro expands to a single
182 expression, you don’t need this extra layer of braces.
183
184 Note that we never *declared* that the macro produces an expression. In fact,
185 this is not determined until we use the macro as an expression. With care, you
186 can write a macro whose expansion works in several contexts. For example,
187 shorthand for a data type could be valid as either an expression or a pattern.
188
189 ## Repetition
190
191 The repetition operator follows two principal rules:
192
193 1. `$(...)*` walks through one "layer" of repetitions, for all of the `$name`s
194 it contains, in lockstep, and
195 2. each `$name` must be under at least as many `$(...)*`s as it was matched
196 against. If it is under more, it’ll be duplicated, as appropriate.
197
198 This baroque macro illustrates the duplication of variables from outer
199 repetition levels.
200
201 ```rust
202 macro_rules! o_O {
203 (
204 $(
205 $x:expr; [ $( $y:expr ),* ]
206 );*
207 ) => {
208 &[ $($( $x + $y ),*),* ]
209 }
210 }
211
212 fn main() {
213 let a: &[i32]
214 = o_O!(10; [1, 2, 3];
215 20; [4, 5, 6]);
216
217 assert_eq!(a, [11, 12, 13, 24, 25, 26]);
218 }
219 ```
220
221 That’s most of the matcher syntax. These examples use `$(...)*`, which is a
222 "zero or more" match. Alternatively you can write `$(...)+` for a "one or
223 more" match. Both forms optionally include a separator, which can be any token
224 except `+` or `*`.
225
226 This system is based on
227 "[Macro-by-Example](https://www.cs.indiana.edu/ftp/techreports/TR206.pdf)"
228 (PDF link).
229
230 # Hygiene
231
232 Some languages implement macros using simple text substitution, which leads to
233 various problems. For example, this C program prints `13` instead of the
234 expected `25`.
235
236 ```text
237 #define FIVE_TIMES(x) 5 * x
238
239 int main() {
240 printf("%d\n", FIVE_TIMES(2 + 3));
241 return 0;
242 }
243 ```
244
245 After expansion we have `5 * 2 + 3`, and multiplication has greater precedence
246 than addition. If you’ve used C macros a lot, you probably know the standard
247 idioms for avoiding this problem, as well as five or six others. In Rust, we
248 don’t have to worry about it.
249
250 ```rust
251 macro_rules! five_times {
252 ($x:expr) => (5 * $x);
253 }
254
255 fn main() {
256 assert_eq!(25, five_times!(2 + 3));
257 }
258 ```
259
260 The metavariable `$x` is parsed as a single expression node, and keeps its
261 place in the syntax tree even after substitution.
262
263 Another common problem in macro systems is ‘variable capture’. Here’s a C
264 macro, using [a GNU C extension] to emulate Rust’s expression blocks.
265
266 [a GNU C extension]: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html
267
268 ```text
269 #define LOG(msg) ({ \
270 int state = get_log_state(); \
271 if (state > 0) { \
272 printf("log(%d): %s\n", state, msg); \
273 } \
274 })
275 ```
276
277 Here’s a simple use case that goes terribly wrong:
278
279 ```text
280 const char *state = "reticulating splines";
281 LOG(state)
282 ```
283
284 This expands to
285
286 ```text
287 const char *state = "reticulating splines";
288 {
289 int state = get_log_state();
290 if (state > 0) {
291 printf("log(%d): %s\n", state, state);
292 }
293 }
294 ```
295
296 The second variable named `state` shadows the first one. This is a problem
297 because the print statement should refer to both of them.
298
299 The equivalent Rust macro has the desired behavior.
300
301 ```rust
302 # fn get_log_state() -> i32 { 3 }
303 macro_rules! log {
304 ($msg:expr) => {{
305 let state: i32 = get_log_state();
306 if state > 0 {
307 println!("log({}): {}", state, $msg);
308 }
309 }};
310 }
311
312 fn main() {
313 let state: &str = "reticulating splines";
314 log!(state);
315 }
316 ```
317
318 This works because Rust has a [hygienic macro system]. Each macro expansion
319 happens in a distinct ‘syntax context’, and each variable is tagged with the
320 syntax context where it was introduced. It’s as though the variable `state`
321 inside `main` is painted a different "color" from the variable `state` inside
322 the macro, and therefore they don’t conflict.
323
324 [hygienic macro system]: https://en.wikipedia.org/wiki/Hygienic_macro
325
326 This also restricts the ability of macros to introduce new bindings at the
327 invocation site. Code such as the following will not work:
328
329 ```rust,ignore
330 macro_rules! foo {
331 () => (let x = 3);
332 }
333
334 fn main() {
335 foo!();
336 println!("{}", x);
337 }
338 ```
339
340 Instead you need to pass the variable name into the invocation, so that it’s
341 tagged with the right syntax context.
342
343 ```rust
344 macro_rules! foo {
345 ($v:ident) => (let $v = 3);
346 }
347
348 fn main() {
349 foo!(x);
350 println!("{}", x);
351 }
352 ```
353
354 This holds for `let` bindings and loop labels, but not for [items][items].
355 So the following code does compile:
356
357 ```rust
358 macro_rules! foo {
359 () => (fn x() { });
360 }
361
362 fn main() {
363 foo!();
364 x();
365 }
366 ```
367
368 [items]: ../reference.html#items
369
370 # Recursive macros
371
372 A macro’s expansion can include more macro invocations, including invocations
373 of the very same macro being expanded. These recursive macros are useful for
374 processing tree-structured input, as illustrated by this (simplistic) HTML
375 shorthand:
376
377 ```rust
378 # #![allow(unused_must_use)]
379 macro_rules! write_html {
380 ($w:expr, ) => (());
381
382 ($w:expr, $e:tt) => (write!($w, "{}", $e));
383
384 ($w:expr, $tag:ident [ $($inner:tt)* ] $($rest:tt)*) => {{
385 write!($w, "<{}>", stringify!($tag));
386 write_html!($w, $($inner)*);
387 write!($w, "</{}>", stringify!($tag));
388 write_html!($w, $($rest)*);
389 }};
390 }
391
392 fn main() {
393 # // FIXME(#21826)
394 use std::fmt::Write;
395 let mut out = String::new();
396
397 write_html!(&mut out,
398 html[
399 head[title["Macros guide"]]
400 body[h1["Macros are the best!"]]
401 ]);
402
403 assert_eq!(out,
404 "<html><head><title>Macros guide</title></head>\
405 <body><h1>Macros are the best!</h1></body></html>");
406 }
407 ```
408
409 # Debugging macro code
410
411 To see the results of expanding macros, run `rustc --pretty expanded`. The
412 output represents a whole crate, so you can also feed it back in to `rustc`,
413 which will sometimes produce better error messages than the original
414 compilation. Note that the `--pretty expanded` output may have a different
415 meaning if multiple variables of the same name (but different syntax contexts)
416 are in play in the same scope. In this case `--pretty expanded,hygiene` will
417 tell you about the syntax contexts.
418
419 `rustc` provides two syntax extensions that help with macro debugging. For now,
420 they are unstable and require feature gates.
421
422 * `log_syntax!(...)` will print its arguments to standard output, at compile
423 time, and "expand" to nothing.
424
425 * `trace_macros!(true)` will enable a compiler message every time a macro is
426 expanded. Use `trace_macros!(false)` later in expansion to turn it off.
427
428 # Syntactic requirements
429
430 Even when Rust code contains un-expanded macros, it can be parsed as a full
431 [syntax tree][ast]. This property can be very useful for editors and other
432 tools that process code. It also has a few consequences for the design of
433 Rust’s macro system.
434
435 [ast]: glossary.html#abstract-syntax-tree
436
437 One consequence is that Rust must determine, when it parses a macro invocation,
438 whether the macro stands in for
439
440 * zero or more items,
441 * zero or more methods,
442 * an expression,
443 * a statement, or
444 * a pattern.
445
446 A macro invocation within a block could stand for some items, or for an
447 expression / statement. Rust uses a simple rule to resolve this ambiguity. A
448 macro invocation that stands for items must be either
449
450 * delimited by curly braces, e.g. `foo! { ... }`, or
451 * terminated by a semicolon, e.g. `foo!(...);`
452
453 Another consequence of pre-expansion parsing is that the macro invocation must
454 consist of valid Rust tokens. Furthermore, parentheses, brackets, and braces
455 must be balanced within a macro invocation. For example, `foo!([)` is
456 forbidden. This allows Rust to know where the macro invocation ends.
457
458 More formally, the macro invocation body must be a sequence of ‘token trees’.
459 A token tree is defined recursively as either
460
461 * a sequence of token trees surrounded by matching `()`, `[]`, or `{}`, or
462 * any other single token.
463
464 Within a matcher, each metavariable has a ‘fragment specifier’, identifying
465 which syntactic form it matches.
466
467 * `ident`: an identifier. Examples: `x`; `foo`.
468 * `path`: a qualified name. Example: `T::SpecialA`.
469 * `expr`: an expression. Examples: `2 + 2`; `if true { 1 } else { 2 }`; `f(42)`.
470 * `ty`: a type. Examples: `i32`; `Vec<(char, String)>`; `&T`.
471 * `pat`: a pattern. Examples: `Some(t)`; `(17, 'a')`; `_`.
472 * `stmt`: a single statement. Example: `let x = 3`.
473 * `block`: a brace-delimited sequence of statements and optionally an expression. Example:
474 `{ log(error, "hi"); return 12; }`.
475 * `item`: an [item][item]. Examples: `fn foo() { }`; `struct Bar;`.
476 * `meta`: a "meta item", as found in attributes. Example: `cfg(target_os = "windows")`.
477 * `tt`: a single token tree.
478
479 There are additional rules regarding the next token after a metavariable:
480
481 * `expr` and `stmt` variables may only be followed by one of: `=> , ;`
482 * `ty` and `path` variables may only be followed by one of: `=> , = | ; : > [ { as where`
483 * `pat` variables may only be followed by one of: `=> , = | if in`
484 * Other variables may be followed by any token.
485
486 These rules provide some flexibility for Rust’s syntax to evolve without
487 breaking existing macros.
488
489 The macro system does not deal with parse ambiguity at all. For example, the
490 grammar `$($i:ident)* $e:expr` will always fail to parse, because the parser would
491 be forced to choose between parsing `$i` and parsing `$e`. Changing the
492 invocation syntax to put a distinctive token in front can solve the problem. In
493 this case, you can write `$(I $i:ident)* E $e:expr`.
494
495 [item]: ../reference.html#items
496
497 # Scoping and macro import/export
498
499 Macros are expanded at an early stage in compilation, before name resolution.
500 One downside is that scoping works differently for macros, compared to other
501 constructs in the language.
502
503 Definition and expansion of macros both happen in a single depth-first,
504 lexical-order traversal of a crate’s source. So a macro defined at module scope
505 is visible to any subsequent code in the same module, which includes the body
506 of any subsequent child `mod` items.
507
508 A macro defined within the body of a single `fn`, or anywhere else not at
509 module scope, is visible only within that item.
510
511 If a module has the `macro_use` attribute, its macros are also visible in its
512 parent module after the child’s `mod` item. If the parent also has `macro_use`
513 then the macros will be visible in the grandparent after the parent’s `mod`
514 item, and so forth.
515
516 The `macro_use` attribute can also appear on `extern crate`. In this context
517 it controls which macros are loaded from the external crate, e.g.
518
519 ```rust,ignore
520 #[macro_use(foo, bar)]
521 extern crate baz;
522 ```
523
524 If the attribute is given simply as `#[macro_use]`, all macros are loaded. If
525 there is no `#[macro_use]` attribute then no macros are loaded. Only macros
526 defined with the `#[macro_export]` attribute may be loaded.
527
528 To load a crate’s macros without linking it into the output, use `#[no_link]`
529 as well.
530
531 An example:
532
533 ```rust
534 macro_rules! m1 { () => (()) }
535
536 // visible here: m1
537
538 mod foo {
539 // visible here: m1
540
541 #[macro_export]
542 macro_rules! m2 { () => (()) }
543
544 // visible here: m1, m2
545 }
546
547 // visible here: m1
548
549 macro_rules! m3 { () => (()) }
550
551 // visible here: m1, m3
552
553 #[macro_use]
554 mod bar {
555 // visible here: m1, m3
556
557 macro_rules! m4 { () => (()) }
558
559 // visible here: m1, m3, m4
560 }
561
562 // visible here: m1, m3, m4
563 # fn main() { }
564 ```
565
566 When this library is loaded with `#[macro_use] extern crate`, only `m2` will
567 be imported.
568
569 The Rust Reference has a [listing of macro-related
570 attributes](../reference.html#macro-related-attributes).
571
572 # The variable `$crate`
573
574 A further difficulty occurs when a macro is used in multiple crates. Say that
575 `mylib` defines
576
577 ```rust
578 pub fn increment(x: u32) -> u32 {
579 x + 1
580 }
581
582 #[macro_export]
583 macro_rules! inc_a {
584 ($x:expr) => ( ::increment($x) )
585 }
586
587 #[macro_export]
588 macro_rules! inc_b {
589 ($x:expr) => ( ::mylib::increment($x) )
590 }
591 # fn main() { }
592 ```
593
594 `inc_a` only works within `mylib`, while `inc_b` only works outside the
595 library. Furthermore, `inc_b` will break if the user imports `mylib` under
596 another name.
597
598 Rust does not (yet) have a hygiene system for crate references, but it does
599 provide a simple workaround for this problem. Within a macro imported from a
600 crate named `foo`, the special macro variable `$crate` will expand to `::foo`.
601 By contrast, when a macro is defined and then used in the same crate, `$crate`
602 will expand to nothing. This means we can write
603
604 ```rust
605 #[macro_export]
606 macro_rules! inc {
607 ($x:expr) => ( $crate::increment($x) )
608 }
609 # fn main() { }
610 ```
611
612 to define a single macro that works both inside and outside our library. The
613 function name will expand to either `::increment` or `::mylib::increment`.
614
615 To keep this system simple and correct, `#[macro_use] extern crate ...` may
616 only appear at the root of your crate, not inside `mod`.
617
618 # The deep end
619
620 The introductory chapter mentioned recursive macros, but it did not give the
621 full story. Recursive macros are useful for another reason: Each recursive
622 invocation gives you another opportunity to pattern-match the macro’s
623 arguments.
624
625 As an extreme example, it is possible, though hardly advisable, to implement
626 the [Bitwise Cyclic Tag](https://esolangs.org/wiki/Bitwise_Cyclic_Tag) automaton
627 within Rust’s macro system.
628
629 ```rust
630 macro_rules! bct {
631 // cmd 0: d ... => ...
632 (0, $($ps:tt),* ; $_d:tt)
633 => (bct!($($ps),*, 0 ; ));
634 (0, $($ps:tt),* ; $_d:tt, $($ds:tt),*)
635 => (bct!($($ps),*, 0 ; $($ds),*));
636
637 // cmd 1p: 1 ... => 1 ... p
638 (1, $p:tt, $($ps:tt),* ; 1)
639 => (bct!($($ps),*, 1, $p ; 1, $p));
640 (1, $p:tt, $($ps:tt),* ; 1, $($ds:tt),*)
641 => (bct!($($ps),*, 1, $p ; 1, $($ds),*, $p));
642
643 // cmd 1p: 0 ... => 0 ...
644 (1, $p:tt, $($ps:tt),* ; $($ds:tt),*)
645 => (bct!($($ps),*, 1, $p ; $($ds),*));
646
647 // halt on empty data string
648 ( $($ps:tt),* ; )
649 => (());
650 }
651 ```
652
653 Exercise: use macros to reduce duplication in the above definition of the
654 `bct!` macro.
655
656 # Common macros
657
658 Here are some common macros you’ll see in Rust code.
659
660 ## panic!
661
662 This macro causes the current thread to panic. You can give it a message
663 to panic with:
664
665 ```rust,no_run
666 panic!("oh no!");
667 ```
668
669 ## vec!
670
671 The `vec!` macro is used throughout the book, so you’ve probably seen it
672 already. It creates `Vec<T>`s with ease:
673
674 ```rust
675 let v = vec![1, 2, 3, 4, 5];
676 ```
677
678 It also lets you make vectors with repeating values. For example, a hundred
679 zeroes:
680
681 ```rust
682 let v = vec![0; 100];
683 ```
684
685 ## assert! and assert_eq!
686
687 These two macros are used in tests. `assert!` takes a boolean. `assert_eq!`
688 takes two values and checks them for equality. `true` passes, `false` `panic!`s.
689 Like this:
690
691 ```rust,no_run
692 // A-ok!
693
694 assert!(true);
695 assert_eq!(5, 3 + 2);
696
697 // nope :(
698
699 assert!(5 < 3);
700 assert_eq!(5, 3);
701 ```
702
703 ## try!
704
705 `try!` is used for error handling. It takes something that can return a
706 `Result<T, E>`, and gives `T` if it’s a `Ok<T>`, and `return`s with the
707 `Err(E)` if it’s that. Like this:
708
709 ```rust,no_run
710 use std::fs::File;
711
712 fn foo() -> std::io::Result<()> {
713 let f = try!(File::create("foo.txt"));
714
715 Ok(())
716 }
717 ```
718
719 This is cleaner than doing this:
720
721 ```rust,no_run
722 use std::fs::File;
723
724 fn foo() -> std::io::Result<()> {
725 let f = File::create("foo.txt");
726
727 let f = match f {
728 Ok(t) => t,
729 Err(e) => return Err(e),
730 };
731
732 Ok(())
733 }
734 ```
735
736 ## unreachable!
737
738 This macro is used when you think some code should never execute:
739
740 ```rust
741 if false {
742 unreachable!();
743 }
744 ```
745
746 Sometimes, the compiler may make you have a different branch that you know
747 will never, ever run. In these cases, use this macro, so that if you end
748 up wrong, you’ll get a `panic!` about it.
749
750 ```rust
751 let x: Option<i32> = None;
752
753 match x {
754 Some(_) => unreachable!(),
755 None => println!("I know x is None!"),
756 }
757 ```
758
759 ## unimplemented!
760
761 The `unimplemented!` macro can be used when you’re trying to get your functions
762 to typecheck, and don’t want to worry about writing out the body of the
763 function. One example of this situation is implementing a trait with multiple
764 required methods, where you want to tackle one at a time. Define the others
765 as `unimplemented!` until you’re ready to write them.
766
767 # Procedural macros
768
769 If Rust’s macro system can’t do what you need, you may want to write a
770 [compiler plugin](compiler-plugins.html) instead. Compared to `macro_rules!`
771 macros, this is significantly more work, the interfaces are much less stable,
772 and bugs can be much harder to track down. In exchange you get the
773 flexibility of running arbitrary Rust code within the compiler. Syntax
774 extension plugins are sometimes called ‘procedural macros’ for this reason.