2 / Copyright (c) 2007 Eric Niebler
4 / Distributed under the Boost Software License, Version 1.0. (See accompanying
5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
8 [/================================================================================]
9 [section:front_end Fronts Ends:
10 Defining Terminals and Non-Terminals of Your EDSL]
11 [/================================================================================]
13 Here is the fun part: designing your own mini-programming language. In this section we'll talk about the nuts and bolts of designing an EDSL interface using Proto. We'll cover the definition of terminals and lazy functions that the users of your EDSL will get to program with. We'll also talk about Proto's expression template-building operator overloads, and about ways to add additional members to expressions within your domain.
15 [/=======================]
16 [section Making Terminals]
17 [/=======================]
19 As we saw with the Calculator example from the Introduction, the simplest way to get an EDSL up and running is simply to define some terminals, as follows.
21 // Define a literal integer Proto expression.
22 proto::terminal<int>::type i = {0};
24 // This creates an expression template.
27 With some terminals and Proto's operator overloads, you can immediately start creating expression templates.
29 Defining terminals -- with aggregate initialization -- can be a little awkward at times. Proto provides an easier-to-use wrapper for literals that can be used to construct Protofied terminal expressions. It's called _literal_.
31 // Define a literal integer Proto expression.
32 proto::literal<int> i = 0;
34 // Proto literals are really just Proto terminal expressions.
35 // For example, this builds a Proto expression template:
38 There is also a _lit_ function for constructing a _literal_ in-place. The above expression can simply be written as:
40 // proto::lit(0) creates an integer terminal expression
45 [/=================================]
46 [section Proto's Operator Overloads]
47 [/=================================]
49 Once we have some Proto terminals, expressions involving those terminals build expression trees for us. Proto defines overloads for each of C++'s overloadable operators in the `boost::proto` namespace. As long as one operand is a Proto expression, the result of the operation is a tree node representing that operation.
51 [note Proto's operator overloads live in the `boost::proto` namespace and are found via ADL (argument-dependent lookup). That is why expressions must be "tainted" with Proto-ness for Proto to be able to build trees out of expressions.]
53 As a result of Proto's operator overloads, we can say:
55 -_1; // OK, build a unary-negate tree node
56 _1 + 42; // OK, build a binary-plus tree node
58 For the most part, this Just Works and you don't need to think about it, but a few operators are special and it can be helpful to know how Proto handles them.
60 [/=========================================================]
61 [heading Assignment, Subscript, and Function Call Operators]
62 [/=========================================================]
64 Proto also overloads `operator=`, `operator[]`, and `operator()`, but these operators are member functions of the expression template rather than free functions in Proto's namespace. The following are valid Proto expressions:
66 _1 = 5; // OK, builds a binary assign tree node
67 _1[6]; // OK, builds a binary subscript tree node
68 _1(); // OK, builds a unary function tree node
69 _1(7); // OK, builds a binary function tree node
70 _1(8,9); // OK, builds a ternary function tree node
73 For the first two lines, assignment and subscript, it should be fairly unsurprising that the resulting expression node should be binary. After all, there are two operands in each expression. It may be surprising at first that what appears to be a function call with no arguments, `_1()`, actually creates an expression node with one child. The child is `_1` itself. Likewise, the expression `_1(7)` has two children: `_1` and `7`.
75 Because these operators can only be defined as member functions, the following expressions are invalid:
78 i = _1; // ERROR: cannot assign _1 to an int
81 p[_1]; // ERROR: cannot use _1 as an index
83 std::sin(_1); // ERROR: cannot call std::sin() with _1
85 Also, C++ has special rules for overloads of `operator->` that make it useless for building expression templates, so Proto does not overload it.
87 [/==============================]
88 [heading The Address-Of Operator]
89 [/==============================]
91 Proto overloads the address-of operator for expression types, so that the following code creates a new unary address-of tree node:
93 &_1; // OK, creates a unary address-of tree node
95 It does /not/ return the address of the `_1` object. However, there is special code in Proto such that a unary address-of node is implicitly convertible to a pointer to its child. In other words, the following code works and does what you might expect, but not in the obvious way:
98 proto::terminal< placeholder<0> >::type
101 _1_type const _1 = {{}};
102 _1_type const * p = &_1; // OK, &_1 implicitly converted
106 [/============================]
107 [section Making Lazy Functions]
108 [/============================]
110 If we limited ourselves to nothing but terminals and operator overloads, our embedded domain-specific languages wouldn't be very expressive. Imagine that we wanted to extend our calculator EDSL with a full suite of math functions like `sin()` and `pow()` that we could invoke lazily as follows.
112 // A calculator expression that takes one argument
113 // and takes the sine of it.
116 We would like the above to create an expression template representing a function invocation. When that expression is evaluated, it should cause the function to be invoked. (At least, that's the meaning of function invocation we'd like the calculator EDSL to have.) You can define `sin` quite simply as follows.
118 // "sin" is a Proto terminal containing a function pointer
119 proto::terminal< double(*)(double) >::type const sin = {&std::sin};
121 In the above, we define `sin` as a Proto terminal containing a pointer to the `std::sin()` function. Now we can use `sin` as a lazy function. The `default_context` that we saw in the Introduction knows how to evaluate lazy functions. Consider the following:
123 double pi = 3.1415926535;
124 proto::default_context ctx;
125 // Create a lazy "sin" invocation and immediately evaluate it
126 std::cout << proto::eval( sin(pi/2), ctx ) << std::endl;
128 The above code prints out:
132 I'm no expert at trigonometry, but that looks right to me.
134 We can write `sin(pi/2)` because the `sin` object, which is a Proto terminal, has an overloaded `operator()()` that builds a node representing a function call invocation. The actual type of `sin(pi/2)` is actually something like this:
136 // The type of the expression sin(pi/2):
138 proto::terminal< double(*)(double) >::type const &
139 proto::result_of::as_child< double const >::type
142 This type further expands to an unsightly node type with a /tag/ type of `proto::tag::function` and two children: the first representing the function to be invoked, and the second representing the argument to the function. (Node tag types describe the operation that created the node. The difference between `a + b` and `a - b` is that the former has tag type `proto::tag::plus` and the latter has tag type `proto::tag::minus`. Tag types are pure compile-time information.)
144 [note In the type computation above, `proto::result_of::as_child<>` is a metafunction that ensures its argument is a Proto expression type. If it isn't one already, it becomes a Proto terminal. We'll learn more about this metafunction, along with _as_child_, its runtime counterpart, [link boost_proto.users_guide.front_end.customizing_expressions_in_your_domain.per_domain_as_child later]. For now, you can forget about it.]
146 It is important to note that there is nothing special about terminals that contain function pointers. /Any/ Proto expression has an overloaded function call operator. Consider:
149 proto::lit(1)(2)(3,4)(5,6,7,8);
151 That may look strange at first. It creates an integer terminal with _lit_, and then invokes it like a function again and again. What does it mean? Who knows?! You get to decide when you define an evaluation context or a transform. But more on that later.
153 [/=======================================]
154 [heading Making Lazy Functions, Continued]
155 [/=======================================]
157 Now, what if we wanted to add a `pow()` function to our calculator EDSL that users could invoke as follows?
159 // A calculator expression that takes one argument
160 // and raises it to the 2nd power
163 The simple technique described above of making `pow` a terminal containing a function pointer doesn't work here. If `pow` is an object, then the expression `pow< 2 >(_1)` is not valid C++. (Well, technically it is; it means, `pow` less than 2, greater than `(_1)`, which is nothing at all like what we want.) `pow` should be a real function template. But it must be an unusual function: one that returns an expression template.
165 With `sin`, we relied on Proto to provide an overloaded `operator()()` to build an expression node with tag type `proto::tag::function` for us. Now we'll need to do so ourselves. As before, the node will have two children: the function to invoke and the function's argument.
167 With `sin`, the function to invoke was a raw function pointer wrapped in a Proto terminal. In the case of `pow`, we want it to be a terminal containing TR1-style function object. This will allow us to parameterize the function on the exponent. Below is the implementation of a simple TR1-style wrapper for the `std::pow` function:
169 // Define a pow_fun function object
173 typedef double result_type;
175 double operator()(double d) const
177 return std::pow(d, Exp);
181 Following the `sin` example, we want `pow< 1 >( pi/2 )` to have a type like this:
183 // The type of the expression pow<1>(pi/2):
185 proto::terminal< pow_fun<1> >::type
186 proto::result_of::as_child< double const >::type
189 We could write a `pow()` function using code like this, but it's verbose and error prone; it's too easy to introduce subtle bugs by forgetting to call _as_child_ where necessary, resulting in code that seems to work but sometimes doesn't. Proto provides a better way to construct expression nodes: _make_expr_.
191 [/=====================================================]
192 [heading Lazy Functions Made Simple With [^make_expr()]]
193 [/=====================================================]
195 Proto provides a helper for building expression templates called _make_expr_. We can concisely define the `pow()` function with it as below.
197 // Define a lazy pow() function for the calculator EDSL.
198 // Can be used as: pow< 2 >(_1)
199 template< int Exp, typename Arg >
200 typename proto::result_of::make_expr<
201 proto::tag::function // Tag type
202 , pow_fun< Exp > // First child (by value)
203 , Arg const & // Second child (by reference)
207 return proto::make_expr<proto::tag::function>(
208 pow_fun<Exp>() // First child (by value)
209 , boost::ref(arg) // Second child (by reference)
213 There are some things to notice about the above code. We use `proto::result_of::make_expr<>` to calculate the return type. The first template parameter is the tag type for the expression node we're building -- in this case, `proto::tag::function`.
215 Subsequent template parameters to `proto::result_of::make_expr<>` represent child nodes. If a child type is not already a Proto expression, it is automatically made into a terminal with _as_child_. A type such as `pow_fun<Exp>` results in terminal that is held by value, whereas a type like `Arg const &` (note the reference) indicates that the result should be held by reference.
217 In the function body is the runtime invocation of _make_expr_. It closely mirrors the return type calculation. _make_expr_ requires you to specify the node's tag type as a template parameter. The arguments to the function become the node's children. When a child should be stored by value, nothing special needs to be done. When a child should be stored by reference, you must use the `boost::ref()` function to wrap the argument.
219 And that's it! _make_expr_ is the lazy person's way to make a lazy funtion.
223 [/=============================================]
224 [section Customizing Expressions in Your Domain]
225 [/=============================================]
227 In this section, we'll learn all about /domains/. In particular, we'll learn:
229 * How to associate Proto expressions with a domain,
230 * How to add members to expressions within a domain,
231 * How to use a /generator/ to post-process all new expressions created in your domain,
232 * How to control which operators are overloaded in a domain,
233 * How to specify capturing policies for child expressions and non-Proto objects, and
234 * How to make expressions from separate domains interoperate.
240 In the [link boost_proto.users_guide.getting_started.hello_calculator Hello Calculator] section, we looked into making calculator expressions directly usable as lambda expressions in calls to STL algorithms, as below:
242 double data[] = {1., 2., 3., 4.};
244 // Use the calculator EDSL to square each element ... HOW?
245 std::transform( data, data + 4, data, _1 * _1 );
247 The difficulty, if you recall, was that by default Proto expressions don't have interesting behaviors of their own. They're just trees. In particular, the expression `_1 * _1` won't have an `operator()` that takes a double and returns a double like `std::transform()` expects -- unless we give it one. To make this work, we needed to define an expression wrapper type that defined the `operator()` member function, and we needed to associate the wrapper with the calculator /domain/.
249 In Proto, the term /domain/ refers to a type that associates expressions in that domain to an expression /generator/. The generator is just a function object that accepts an expression and does something to it, like wrapping it in an expression wrapper.
251 You can also use a domain to associate expressions with a grammar. When you specify a domain's grammar, Proto ensures that all the expressions it generates in that domain conform to the domain's grammar. It does that by disabling any operator overloads that would create invalid expressions.
255 [/==================================================]
256 [section:extends The [^extends<>] Expression Wrapper]
257 [/==================================================]
259 The first step to giving your calculator expressions extra behaviors is to define a calculator domain. All expressions within the calculator domain will be imbued with calculator-ness, as we'll see.
261 // A type to be used as a domain tag (to be defined below)
262 struct calculator_domain;
264 We use this domain type when extending the _expr_ type, which we do with the _extends_ class template. Here is our expression wrapper, which imbues an expression with calculator-ness. It is described below.
266 // The calculator<> expression wrapper makes expressions
268 template< typename Expr >
270 : proto::extends< Expr, calculator< Expr >, calculator_domain >
273 proto::extends< Expr, calculator< Expr >, calculator_domain >
276 calculator( Expr const &expr = Expr() )
280 // This is usually needed because by default, the compiler-
281 // generated assignment operator hides extends<>::operator=
282 BOOST_PROTO_EXTENDS_USING_ASSIGN(calculator)
284 typedef double result_type;
286 // Hide base_type::operator() by defining our own which
287 // evaluates the calculator expression with a calculator context.
288 result_type operator()( double d1 = 0.0, double d2 = 0.0 ) const
290 // As defined in the Hello Calculator section.
291 calculator_context ctx;
293 // ctx.args is a vector<double> that holds the values
294 // with which we replace the placeholders (e.g., _1 and _2)
295 // in the expression.
296 ctx.args.push_back( d1 ); // _1 gets the value of d1
297 ctx.args.push_back( d2 ); // _2 gets the value of d2
299 return proto::eval(*this, ctx ); // evaluate the expression
303 We want calculator expressions to be function objects, so we have to define an `operator()` that takes and returns doubles. The `calculator<>` wrapper above does that with the help of the _extends_ template. The first template to _extends_ parameter is the expression type we are extending. The second is the type of the wrapped expression. The third parameter is the domain that this wrapper is associated with. A wrapper type like `calculator<>` that inherits from _extends_ behaves just like the expression type it has extended, with any additional behaviors you choose to give it.
305 [note [*Why not just inherit from [^proto::expr<>]?]
307 You might be thinking that this expression extension business is unnecessarily complicated. After all, isn't this why C++ supports inheritance? Why can't [^calculator<Expr>] just inherit from [^Expr] directly? The reason is because [^Expr], which presumably is an instantiation of _expr_, has expression template-building operator overloads that will be incorrect for derived types. They will store `*this` by reference to `proto::expr<>`, effectively slicing off any derived parts. _extends_ gives your derived types operator overloads that don't slice off your additional members.]
309 Although not strictly necessary in this case, we bring `extends<>::operator=` into scope with the `BOOST_PROTO_EXTENDS_USING_ASSIGN()` macro. This is really only necessary if you want expressions like `_1 = 3` to create a lazily evaluated assignment. _extends_ defines the appropriate `operator=` for you, but the compiler-generated `calculator<>::operator=` will hide it unless you make it available with the macro.
311 Note that in the implementation of `calculator<>::operator()`, we evaluate the expression with the `calculator_context` we defined earlier. As we saw before, the context is what gives the operators their meaning. In the case of the calculator, the context is also what defines the meaning of the placeholder terminals.
313 Now that we have defined the `calculator<>` expression wrapper, we need to wrap the placeholders to imbue them with calculator-ness:
315 calculator< proto::terminal< placeholder<0> >::type > const _1;
316 calculator< proto::terminal< placeholder<1> >::type > const _2;
318 [/=======================================================]
319 [heading Retaining POD-ness with [^BOOST_PROTO_EXTENDS()]]
320 [/=======================================================]
322 To use _extends_, your extension type must derive from _extends_. Unfortunately, that means that your extension type is no longer POD and its instances cannot be /statically initialized/. (See the [link boost_proto.appendices.rationale.static_initialization Static
323 Initialization] section in the [link boost_proto.appendices.rationale Rationale] appendix for why this matters.) In particular, as defined above, the global placeholder objects `_1` and `_2` will need to be initialized at runtime, which could lead to subtle order of initialization bugs.
325 There is another way to make an expression extension that doesn't sacrifice POD-ness : the _EXTENDS_ macro. You can use it much like you use _extends_. We can use _EXTENDS_ to keep `calculator<>` a POD and our placeholders statically initialized.
327 // The calculator<> expression wrapper makes expressions
329 template< typename Expr >
332 // Use BOOST_PROTO_EXTENDS() instead of proto::extends<> to
333 // make this type a Proto expression extension.
334 BOOST_PROTO_EXTENDS(Expr, calculator<Expr>, calculator_domain)
336 typedef double result_type;
338 result_type operator()( double d1 = 0.0, double d2 = 0.0 ) const
340 /* ... as before ... */
344 With the new `calculator<>` type, we can redefine our placeholders to be statically initialized:
346 calculator< proto::terminal< placeholder<0> >::type > const _1 = {{{}}};
347 calculator< proto::terminal< placeholder<1> >::type > const _2 = {{{}}};
349 We need to make one additional small change to accommodate the POD-ness of our expression extension, which we'll describe below in the section on expression generators.
351 What does _EXTENDS_ do? It defines a data member of the expression type being extended; some nested typedefs that Proto requires; `operator=`, `operator[]` and `operator()` overloads for building expression templates; and a nested `result<>` template for calculating the return type of `operator()`. In this case, however, the `operator()` overloads and the `result<>` template are not needed because we are defining our own `operator()` in the `calculator<>` type. Proto provides additional macros for finer control over which member functions are defined. We could improve our `calculator<>` type as follows:
353 // The calculator<> expression wrapper makes expressions
355 template< typename Expr >
358 // Use BOOST_PROTO_BASIC_EXTENDS() instead of proto::extends<> to
359 // make this type a Proto expression extension:
360 BOOST_PROTO_BASIC_EXTENDS(Expr, calculator<Expr>, calculator_domain)
362 // Define operator[] to build expression templates:
363 BOOST_PROTO_EXTENDS_SUBSCRIPT()
365 // Define operator= to build expression templates:
366 BOOST_PROTO_EXTENDS_ASSIGN()
368 typedef double result_type;
370 result_type operator()( double d1 = 0.0, double d2 = 0.0 ) const
372 /* ... as before ... */
376 Notice that we are now using _BASIC_EXTENDS_ instead of _EXTENDS_. This just adds the data member and the nested typedefs but not any of the overloaded operators. Those are added separately with _EXTENDS_ASSIGN_ and _EXTENDS_SUBSCRIPT_. We are leaving out the function call operator and the nested `result<>` template that could have been defined with Proto's _EXTENDS_FUNCTION_ macro.
378 In summary, here are the macros you can use to define expression extensions, and a brief description of each.
380 [def __expression__ [~expression]]
381 [def __extension__ [~extension]]
382 [def __domain__ [~domain]]
383 [def __extends__ [macroref BOOST_PROTO_EXTENDS]]
384 [def __basic_extends__ [macroref BOOST_PROTO_BASIC_EXTENDS]]
386 [table Expression Extension Macros
389 [[``__basic_extends__(
394 [Defines a data member of type `__expression__` and some nested typedefs that Proto requires.]]
396 [Defines `operator=`. Only valid when preceded by _BASIC_EXTENDS_.]]
397 [[_EXTENDS_SUBSCRIPT_]
398 [Defines `operator[]`. Only valid when preceded by _BASIC_EXTENDS_.]]
399 [[_EXTENDS_FUNCTION_]
400 [Defines `operator()` and a nested `result<>` template for return type calculation. Only valid when preceded by _BASIC_EXTENDS_.]]
407 __basic_extends__(__expression__, __extension__, __domain__)
410 _EXTENDS_FUNCTION_``]]
413 [warning [*Argument-Dependent Lookup and _EXTENDS_]
415 Proto's operator overloads are defined in the `boost::proto` namespace and are found by argument-dependent lookup (ADL). This usually just works because expressions are made up of types that live in the `boost::proto` namespace. However, sometimes when you use _EXTENDS_ that is not the case. Consider:
421 typename proto::terminal<std::complex<T> >::type
423 , proto::default_domain
429 my_complex<int> c0, c1;
431 c0 + c1; // ERROR: operator+ not found
435 The problem has to do with how argument-dependent lookup works. The type `my_complex<int>` is not associated in any way with the `boost::proto` namespace, so the operators defined there are not considered. (Had we inherited from _extends_ instead of used _EXTENDS_, we would have avoided the problem because inheriting from a type in `boost::proto` namespace is enough to get ADL to kick in.)
437 So what can we do? By adding an extra dummy template parameter that defaults to a type in the `boost::proto` namespace, we can trick ADL into finding the right operator overloads. The solution looks like this:
439 `` template<class T, class Dummy = proto::is_proto_expr>
443 typename proto::terminal<std::complex<T> >::type
445 , proto::default_domain
451 my_complex<int> c0, c1;
453 c0 + c1; // OK, operator+ found now!
457 The type [classref boost::proto::is_proto_expr `proto::is_proto_expr`] is nothing but an empty struct, but by making it a template parameter we make `boost::proto` an associated namespace of `my_complex<int>`. Now ADL can successfully find Proto's operator overloads.
462 [/============================]
463 [section Expression Generators]
464 [/============================]
466 The last thing that remains to be done is to tell Proto that it needs to wrap all of our calculator expressions in our `calculator<>` wrapper. We have already wrapped the placeholders, but we want /all/ expressions that involve the calculator placeholders to be calculators. We can do that by specifying an expression generator when we define our `calculator_domain`, as follows:
468 // Define the calculator_domain we forward-declared above.
469 // Specify that all expression in this domain should be wrapped
470 // in the calculator<> expression wrapper.
471 struct calculator_domain
472 : proto::domain< proto::generator< calculator > >
475 The first template parameter to `proto::domain<>` is the generator. "Generator" is just a fancy name for a function object that accepts an expression and does something to it. `proto::generator<>` is a very simple one --- it wraps an expression in the wrapper you specify. `proto::domain<>` inherits from its generator parameter, so all domains are themselves function objects.
477 If we used _EXTENDS_ to keep our expression extension type POD, then we need to use `proto::pod_generator<>` instead of `proto::generator<>`, as follows:
479 // If calculator<> uses BOOST_PROTO_EXTENDS() instead of
480 // use proto::extends<>, use proto::pod_generator<> instead
481 // of proto::generator<>.
482 struct calculator_domain
483 : proto::domain< proto::pod_generator< calculator > >
486 [def __Domain__ [~Domain]]
488 After Proto has calculated a new expression type, it checks the domains of the child expressions. They must match. Assuming they do, Proto creates the new expression and passes it to `__Domain__::operator()` for any additional processing. If we don't specify a generator, the new expression gets passed through unchanged. But since we've specified a generator above, `calculator_domain::operator()` returns `calculator<>` objects.
490 Now we can use calculator expressions as function objects to STL algorithms, as follows:
492 double data[] = {1., 2., 3., 4.};
494 // Use the calculator EDSL to square each element ... WORKS! :-)
495 std::transform( data, data + 4, data, _1 * _1 );
499 [/==========================================================]
500 [section:inhibiting_overloads Controlling Operator Overloads]
501 [/==========================================================]
503 By default, Proto defines every possible operator overload for Protofied
504 expressions. This makes it simple to bang together an EDSL. In some cases, however, the presence of Proto's promiscuous overloads can lead to confusion or worse. When that happens, you'll have to disable some of Proto's overloaded operators. That is done by defining the grammar for your domain and specifying it as the second parameter of the _domain_ template.
506 In the [link boost_proto.users_guide.getting_started.hello_calculator Hello Calculator] section, we saw an example of a Proto grammar, which is repeated here:
508 // Define the grammar of calculator expressions
509 struct calculator_grammar
511 proto::plus< calculator_grammar, calculator_grammar >
512 , proto::minus< calculator_grammar, calculator_grammar >
513 , proto::multiplies< calculator_grammar, calculator_grammar >
514 , proto::divides< calculator_grammar, calculator_grammar >
515 , proto::terminal< proto::_ >
519 We'll have much more to say about grammars in subsequent sections, but for now, we'll just say that the `calculator_grammar` struct describes a subset of all expression types -- the subset that comprise valid calculator expressions. We would like to prohibit Proto from creating a calculator expression that does not conform to this grammar. We do that by changing the definition of the `calculator_domain` struct.
521 [def __calculator_grammar__ [*calculator_grammar]]
523 // Define the calculator_domain. Expressions in the calculator
524 // domain are wrapped in the calculator<> wrapper, and they must
525 // conform to the calculator_grammar:
526 struct calculator_domain
527 : proto::domain< proto::generator< calculator >, __calculator_grammar__ >
530 The only new addition is `calculator_grammar` as the second template parameter to the _domain_ template. That has the effect of disabling any of Proto's operator overloads that would create an invalid calculator expression.
532 Another common use for this feature would be to disable Proto's unary `operator&` overload. It may be surprising for users of your EDSL that they cannot take the address of their expressions! You can very easily disable Proto's unary `operator&` overload for your domain with a very simple grammar, as below:
534 // For expressions in my_domain, disable Proto's
535 // unary address-of operator.
538 proto::generator< my_wrapper >
539 // A simple grammar that matches any expression that
540 // is not a unary address-of expression.
541 , proto::not_< proto::address_of< _ > >
545 The type `proto::not_< proto::address_of< _ > >` is a very simple grammar that matches all expressions except unary address-of expressions. In the section describing Proto's intermediate form, we'll have much more to say about grammars.
549 [/=========================================================================]
550 [section:per_domain_as_child Controlling How Child Expressions Are Captured]
551 [/=========================================================================]
553 [note This is an advanced topic. Feel free to skip this if you're just getting started with Proto.]
555 Proto's operator overloads build expressions from sub-expressions. The sub-expressions become children of the new expression. By default, the children are stored in the parent by reference. This section describes how to change that default.
557 [/-----------------------------------------]
558 [heading Primer: [^as_child] vs. [^as_expr]]
559 [/-----------------------------------------]
561 Proto lets you independently customize the behavior of _as_child_ and _as_expr_. Both accept an object [^x] and return a Proto expression by turning [^x] it into a Proto terminal if necessary. Although similar, the two functions are used in different situations and have subtly different behavior by default. It's important to understand the difference so that you know which to customize to achieve the behavior you want.
563 To wit: _as_expr_ is typically used by /you/ to turn an object into a Proto expression that is to be held in a local variable, as so:
565 auto l = proto::as_expr(x); // Turn x into a Proto expression, hold the result in a local
567 The above works regardless of whether `x` is already a Proto expression or not. The object `l` is guaranteed to be a valid Proto expression. If `x` is a non-Proto object, it is turned into a terminal expression that holds `x` /by value/.[footnote It's not always possible to hold something by value. By default, _as_expr_ makes an exception for functions, abstract types, and iostreams (types derived from `std::ios_base`). These objects are held by reference. All others are held by value, even arrays.] If `x` is a Proto object already, _as_expr_ returns it /by value/ unmodified.
569 In contrast, _as_child_ is used internally by Proto to pre-process objects before making them children of another expression. Since it's internal to Proto, you don't see it explicitly, but it's there behind the scenes in expressions like this:
571 x + y; // Consider that y is a Proto expression, but x may or may not be.
573 In this case, Proto builds a plus node from the two children. Both are pre-processed by passing them to _as_child_ before making them children of the new node. If `x` is not a Proto expression, it becomes one by being wrapped in a Proto terminal that holds it /by reference/. If `x` is already a Proto expression, _as_child_ returns it /by reference/ unmodified. Contrast this with the above description for _as_expr_.
575 The table below summarizes the above description.
577 [table proto::as_expr() vs. proto::as_child()
578 [[[*Function]] [[*When [^t] is not a Proto expr...]] [[*When [^t] is a Proto expr...]]]
579 [[[^proto::as_expr(t)]] [Return (by value) a new Proto terminal holding [^t] by value.] [Return [^t] by value unmodified.]]
580 [[[^proto::as_child(t)]] [Return (by value) a new Proto terminal holding [^t] by reference.] [Return [^t] by reference unmodified.]]
583 [note There is one important place where Proto uses both `as_expr` /and/ `as_child`: _make_expr_. The _make_expr_ function requires you to specify for each child whether it should be held by value or by reference. Proto uses _as_expr_ to pre-process the children to be held by value, and _as_child_ for the ones to be held by reference.]
585 Now that you know what _as_child_ and _as_expr_ are, where they are used, and what they do by default, you may decide that one or both of these functions should have different behavior for your domain. For instance, given the above description of _as_child_, the following code is always wrong:
587 proto::literal<int> i(0);
588 auto l = i + 42; // This is WRONG! Don't do this.
590 Why is this wrong? Because _as_child_ will turn the integer literal 42 into a Proto terminal that holds a reference to a temporary integer initialized with 42. The lifetime of that temporary ends at the semicolon, guaranteeing that the local `l` is left holding a dangling reference to a deceased integer. What to do? One answer is to use _deep_copy_. Another is to customize the behavior of _as_child_ for your domain. Read on for the details.
592 [/-----------------------------]
593 [heading Per-Domain [^as_child]]
594 [/-----------------------------]
596 To control how Proto builds expressions out of sub-expressions in your domain, define your domain as usual, and then define a nested `as_child<>` class template within it, as follows:
598 [def __unspecified_expression_type__ ['[^unspecified-Proto-expr-type]]]
599 [def __unspecified_expression__ ['[^unspecified-Proto-expr-object]]]
602 : proto::domain< my_generator, my_grammar >
604 // Here is where you define how Proto should handle
605 // sub-expressions that are about to be glommed into
606 // a larger expression.
607 template< typename T >
610 typedef __unspecified_expression_type__ result_type;
612 result_type operator()( T & t ) const
614 return __unspecified_expression__;
619 There's one important thing to note: in the above code, the template parameter [^T] may or may not be a Proto expression type, but the result /must/ be a Proto expression type, or a reference to one. That means that most user-defined [^as_child<>] templates will need to check whether [^T] is an expression or not (using _is_expr_), and then turn non-expressions into Proto terminals by wrapping them as `proto::terminal< /* ... */ >::type` or equivalent.
621 [/----------------------------]
622 [heading Per-Domain [^as_expr]]
623 [/----------------------------]
625 Although less common, Proto also lets you customize the behavior of _as_expr_ on a per-domain basis. The technique is identical to that for [^as_child]. See below:
628 : proto::domain< my_generator, my_grammar >
630 // Here is where you define how Proto should handle
631 // objects that are to be turned into expressions
632 // fit for storage in local variables.
633 template< typename T >
636 typedef __unspecified_expression_type__ result_type;
638 result_type operator()( T & t ) const
640 return __unspecified_expression__;
646 [/--------------------------------------------]
647 [heading Making Proto Expressions [^auto]-safe]
648 [/--------------------------------------------]
650 Let's look again at the problem described above involving the C++11 `auto` keyword and the default behavior of _as_child_.
652 proto::literal<int> i(0);
653 auto l = i + 42; // This is WRONG! Don't do this.
655 Recall that the problem is the lifetime of the temporary integer created to hold the value 42. The local `l` will be left holding a dangling reference to it after its lifetime is over. What if we want Proto to make expressions safe to store this way in local variables? We can do so very easily by making _as_child_ behave just like _as_expr_. The following code achieves this:
657 template< typename E >
661 : proto::pod_generator< my_expr >
665 : proto::domain< my_generator >
667 // Make as_child() behave like as_expr() in my_domain.
668 // (proto_base_domain is a typedef for proto::domain< my_generator >
669 // that is defined in proto::domain<>.)
670 template< typename T >
672 : proto_base_domain::as_expr< T >
676 template< typename E >
679 BOOST_PROTO_EXTENDS( E, my_expr< E >, my_domain )
684 proto::literal< int, my_domain > i(0);
685 auto l = i + 42; // OK! Everything is stored by value here.
687 Notice that `my_domain::as_child<>` simply defers to the default implementation of `as_expr<>` found in _domain_. By simply cross-wiring our domain's `as_child<>` to `as_expr<>`, we guarantee that all terminals that can be held by value are, and that all child expressions are also held by value. This increases copying and may incur a runtime performance cost, but it eliminates any spector of lifetime management issues.
689 For another example, see the definition of `lldomain` in [^libs/proto/example/lambda.hpp]. That example is a complete reimplementation of the Boost Lambda Library (BLL) on top of Boost.Proto. The function objects the BLL generates are safe to be stored in local variables. To emulate this with Proto, the `lldomain` cross-wires `as_child<>` to `as_expr<>` as above, but with one extra twist: objects with array type are also stored by reference. Check it out.
693 [/======================================================]
694 [section:subdomains EDSL Interoperatability: Sub-Domains]
695 [/======================================================]
697 [note This is an advanced topic. Feel free to skip this if you're just getting started with Proto.]
699 The ability to /compose/ different EDSLs is one of their most exciting features. Consider how you build a parser using yacc. You write your grammar rules in yacc's domain-specific language. Then you embed semantic actions written in C within your grammar. Boost's Spirit parser generator gives you the same ability. You write grammar rules using Spirit.Qi and embed semantic actions using the Phoenix library. Phoenix and Spirit are both Proto-based domain-specific languages with their own distinct syntax and semantics. But you can freely embed Phoenix expressions within Spirit expressions. This section describes Proto's /sub-domain/ feature that lets you define families of interoperable domains.
701 [/======================]
702 [heading Dueling Domains]
703 [/======================]
705 When you try to create an expression from two sub-expressions in different domains, what is the domain of the resulting expression? This is the fundamental problem that is addressed by sub-domains. Consider the following code:
707 #include <boost/proto/proto.hpp>
708 namespace proto = boost::proto;
710 // Forward-declare two expression wrappers
711 template<typename E> struct spirit_expr;
712 template<typename E> struct phoenix_expr;
714 // Define two domains
715 struct spirit_domain : proto::domain<proto::generator<spirit_expr> > {};
716 struct phoenix_domain : proto::domain<proto::generator<phoenix_expr> > {};
718 // Implement the two expression wrappers
721 : proto::extends<E, spirit_expr<E>, spirit_domain>
723 spirit_expr(E const &e = E()) : spirit_expr::proto_extends(e) {}
728 : proto::extends<E, phoenix_expr<E>, phoenix_domain>
730 phoenix_expr(E const &e = E()) : phoenix_expr::proto_extends(e) {}
735 proto::literal<int, spirit_domain> sp(0);
736 proto::literal<int, phoenix_domain> phx(0);
738 // Whoops! What does it mean to add two expressions in different domains?
742 Above, we define two domains called `spirit_domain` and `phoenix_domain` and declare two int literals in each. Then we try to compose them into a larger expression using Proto's binary plus operator, and it fails. Proto can't figure out whether the resulting expression should be in the Spirit domain or the Phoenix domain, and thus whether it should be an instance of `spirit_expr<>` or `phoenix_expr<>`. We have to tell Proto how to resolve the conflict. We can do that by declaring that Phoenix is a sub-domain of Spirit as in the following definition of `phoenix_domain`:
744 [def __spirit_domain__ [*spirit_domain]]
746 // Declare that phoenix_domain is a sub-domain of spirit_domain
747 struct phoenix_domain
748 : proto::domain<proto::generator<phoenix_expr>, proto::_, __spirit_domain__>
751 The third template parameter to _domain_ is the super-domain. By defining `phoenix_domain` as above, we are saying that Phoenix expressions can be combined with Spirit expressions, and that when that happens, the resulting expression should be a Spirit expression.
753 [note If you are wondering what the purpose of `proto::_` is in the definition of `phoenix_domain` above, recall that the second template parameter to _domain_ is the domain's grammar. ["`proto::_`] is the default and signifies that the domain places no restrictions on the expressions that are valid within it.]
755 [/------------------------]
756 [heading Domain Resolution]
757 [/------------------------]
759 When there are multiple domains in play within a given expression, Proto uses some rules to figure out which domain "wins". The rules are loosely modeled on the rules for C++ inheritance. `Phoenix_domain` is a sub-domain of `spirit_domain`. You can liken that to a derived/base relationship that gives Phoenix expressions a kind of implicit conversion to Spirit expressions. And since Phoenix expressions can be "converted" to Spirit expressions, they can be freely combined with Spirit expressions and the result is a Spirit expression.
761 [note Super- and sub-domains are not actually implemented using inheritance. This is only a helpful mental model.]
763 The analogy with inheritance holds even in the case of three domains when two are sub-domains of the third. Imagine another domain called `foobar_domain` that was also a sub-domain of `spirit_domain`. Expressions in the `foobar_domain` could be combined with expressions in the `phoenix_domain` and the resulting expression would be in the `spirit_domain`. That's because expressions in the two sub-domains both have "conversions" to the super-domain, so the operation is allowed and the super-domain wins.
765 [/-------------------------]
766 [heading The Default Domain]
767 [/-------------------------]
769 When you don't assign a Proto expression to a particular domain, Proto considers it a member of the so-called default domain, `proto::default_domain`. Even non-Proto objects are treated as terminals in the default domain. Consider:
773 proto::literal<int, spirit_domain> sp(0);
775 // Add 1 to a spirit expression. Result is a spirit expression.
779 Expressions in the default domain (or non-expressions like [^1]) have a kind of implicit conversion to expressions every other domain type. What's more, you can define your domain to be a sub-domain of the default domain. In so doing, you give expressions in your domain conversions to expressions in every other domain. This is like a ["free love] domain, because it will freely mix with all other domains.
781 Let's think again about the Phoenix EDSL. Since it provides generally useful lambda functionality, it's reasonable to assume that lots of other EDSLs besides Spirit might want the ability to embed Phoenix expressions. In other words, `phoenix_domain` should be a sub-domain of `proto::default_domain`, not `spirit_domain`:
783 // Declare that phoenix_domain is a sub-domain of proto::default_domain
784 struct phoenix_domain
785 : proto::domain<proto::generator<phoenix_expr>, proto::_, proto::default_domain>
788 That's much better. Phoenix expressions can now be put anywhere.
790 [/-------------------------]
791 [heading Sub-Domain Summary]
792 [/-------------------------]
794 Use Proto sub-domains to make it possible to mix expressions from multiple domains. And when you want expressions in your domain to freely combine with /all/ expressions, make it a sub-domain of `proto::default_domain`.
800 [section:define_operators Adapting Existing Types to Proto]
802 The preceding discussions of defining Proto front ends have all made a big assumption: that you have the luxury of defining everything from scratch. What happens if you have existing types, say a matrix type and a vector type, that you would like to treat as if they were Proto terminals? Proto usually trades only in its own expression types, but with _DEFINE_OPERATORS_, it can accomodate your custom terminal types, too.
804 Let's say, for instance, that you have the following types and that you can't modify then to make them ["native] Proto terminal types.
809 struct matrix { /*...*/ };
812 struct vector { /*...*/ };
815 You can non-intrusively make objects of these types Proto terminals by defining the proper operator overloads using _DEFINE_OPERATORS_. The basic procedure is as follows:
817 # Define a trait that returns true for your types and false for all others.
818 # Reopen the namespace of your types and use _DEFINE_OPERATORS_ to define a set of
819 operator overloads, passing the name of the trait as the first macro parameter,
820 and the name of a Proto domain (e.g., _default_domain_) as the second.
822 The following code demonstrates how it works.
831 // OK, "matrix" is a custom terminal type
833 struct is_terminal<matrix>
837 // OK, "vector" is a custom terminal type
839 struct is_terminal<vector>
843 // Define all the operator overloads to construct Proto
844 // expression templates, treating "matrix" and "vector"
845 // objects as if they were Proto terminals.
846 BOOST_PROTO_DEFINE_OPERATORS(is_terminal, proto::default_domain)
849 The invocation of the _DEFINE_OPERATORS_ macro defines a complete set of operator overloads that treat `matrix` and `vector` objects as if they were Proto terminals. And since the operators are defined in the same namespace as the `matrix` and `vector` types, the operators will be found by argument-dependent lookup. With the code above, we can now construct expression templates with matrices and vectors, as shown below.
853 proto::literal<int> i(0);
855 m1 * 1; // custom terminal and literals are OK
856 m1 * i; // custom terminal and Proto expressions are OK
857 m1 * v1; // two custom terminals are OK, too.
861 [/=======================================================================]
862 [section:code_repetition Generating Repetitive Code with the Preprocessor]
863 [/=======================================================================]
865 Sometimes as an EDSL designer, to make the lives of your users easy, you have to make your own life hard. Giving your users natural and flexible syntax often involves writing large numbers of repetitive function overloads. It can be enough to give you repetitive stress injury! Before you hurt yourself, check out the macros Proto provides for automating many repetitive code-generation chores.
867 Imagine that we are writing a lambda EDSL, and we would like to enable syntax for constructing temporary objects of any type using the following syntax:
869 // A lambda expression that takes two arguments and
870 // uses them to construct a temporary std::complex<>
871 construct< std::complex<int> >( _1, _2 )
873 For the sake of the discussion, imagine that we already have a function object template `construct_impl<>` that accepts arguments and constructs new objects from them. We would want the above lambda expression to be equivalent to the following:
875 // The above lambda expression should be roughly equivalent
877 proto::make_expr<proto::tag::function>(
878 construct_impl<std::complex<int> >() // The function to invoke lazily
879 , boost::ref(_1) // The first argument to the function
880 , boost::ref(_2) // The second argument to the function
883 We can define our `construct()` function template as follows:
885 template<typename T, typename A0, typename A1>
886 typename proto::result_of::make_expr<
892 construct(A0 const &a0, A1 const &a1)
894 return proto::make_expr<proto::tag::function>(
901 This works for two arguments, but we would like it to work for any number of arguments, up to (_MAX_ARITY_ - 1). (Why "- 1"? Because one child is taken up by the `construct_impl<T>()` terminal leaving room for only (_MAX_ARITY_ - 1) other children.)
903 For cases like this, Proto provides the _REPEAT_ and _REPEAT_FROM_TO_ macros. To use it, we turn the function definition above into a macro as follows:
905 #define M0(N, typename_A, A_const_ref, A_const_ref_a, ref_a) \
906 template<typename T, typename_A(N)> \
907 typename proto::result_of::make_expr< \
908 proto::tag::function \
909 , construct_impl<T> \
912 construct(A_const_ref_a(N)) \
914 return proto::make_expr<proto::tag::function>( \
915 construct_impl<T>() \
920 Notice that we turned the function into a macro that takes 5 arguments. The first is the current iteration number. The rest are the names of other macros that generate different sequences. For instance, Proto passes as the second parameter the name of a macro that will expand to `typename A0, typename A1, ...`.
922 Now that we have turned our function into a macro, we can pass the macro to _REPEAT_FROM_TO_. Proto will invoke it iteratively, generating all the function overloads for us.
924 // Generate overloads of construct() that accept from
925 // 1 to BOOST_PROTO_MAX_ARITY-1 arguments:
926 BOOST_PROTO_REPEAT_FROM_TO(1, BOOST_PROTO_MAX_ARITY, M0)
929 [/============================]
930 [heading Non-Default Sequences]
931 [/============================]
933 As mentioned above, Proto passes as the last 4 arguments to your macro the names of other macros that generate various sequences. The macros _REPEAT_ and _REPEAT_FROM_TO_ select defaults for these parameters. If the defaults do not meet your needs, you can use _REPEAT_EX_ and _REPEAT_FROM_TO_EX_ and pass different macros that generate different sequences. Proto defines a number of such macros for use as parameters to _REPEAT_EX_ and _REPEAT_FROM_TO_EX_. Check the reference section for [headerref boost/proto/repeat.hpp] for all the details.
935 Also, check out _LOCAL_ITERATE_. It works similarly to _REPEAT_ and friends, but it can be easier to use when you want to change one macro argument and accept defaults for the others.