]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | [/============================================================================== |
2 | Copyright (C) 2001-2011 Hartmut Kaiser | |
3 | Copyright (C) 2001-2011 Joel de Guzman | |
4 | ||
5 | Distributed under the Boost Software License, Version 1.0. (See accompanying | |
6 | file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) | |
7 | ===============================================================================/] | |
8 | ||
9 | [section:lexer_concepts Lexer Concepts] | |
10 | ||
11 | __lex__ components fall into a couple of generalized __concepts__. The | |
12 | /Lexer/ is the most fundamental concept. All __lex__ components are | |
13 | models of the /Lexer/ concept. /PrimitiveLexer/, /UnaryLexer/, | |
14 | and /NaryLexer/ are all refinements of the /Lexer/ concept. | |
15 | ||
16 | The following sections provide details on these concepts. | |
17 | ||
18 | [/////////////////////////////////////////////////////////////////////////////] | |
19 | [section Lexer] | |
20 | ||
21 | [heading Description] | |
22 | ||
23 | The /Lexer/ is the most fundamental concept. A Lexer has a member | |
24 | function, `collect`, that accepts a token definition container `Def`, and a | |
25 | the name of the lexer state the token definitions of the lexer component need | |
26 | to be added to (a string). It doesn't return anything (return type is `void`). | |
27 | Each Lexer can represent a specific pattern or algorithm, or it | |
28 | can be a more complex lexer component formed as a composition of other Lexer's. | |
29 | Additionally, a Lexer exposes a member `add_actions`, that accepts the token | |
30 | definition container `Def`, while returning nothing (again, the returned type | |
31 | is `void`). | |
32 | ||
33 | [variablelist Notation | |
34 | [[`l`] [A `Lexer`.]] | |
35 | [[`L`] [A `Lexer` type.]] | |
36 | [[`Def`] [A token definition container type.]] | |
37 | [[`State`] [A type used to represent lexer state names.]] | |
38 | ] | |
39 | ||
40 | [heading Valid Expressions] | |
41 | ||
42 | In the expressions below, the behavior of the lexer component, `l`, is left | |
43 | unspecified in the base `Lexer` concept. These are specified in subsequent, | |
44 | more refined concepts and by the actual models thereof. | |
45 | ||
46 | For any Lexer the following expressions must be valid: | |
47 | ||
48 | [table | |
49 | [[Expression] [Semantics] [Return type]] | |
50 | [[`l.collect(def, state, targetstate)`] | |
51 | [Add all token definitions provided | |
52 | by this Lexer instance to the lexer | |
53 | state `state` of the token definition | |
54 | container `def`. After matching this token, the | |
55 | lexer should be switched into the state | |
56 | `targetstate` (optional)] [`void`]] | |
57 | [[`l.add_actions(def)`] [Add all semantic actions provided | |
58 | by this Lexer instance to the token | |
59 | definition container `def`.] [`void`]] | |
60 | ] | |
61 | ||
62 | [heading Type Expressions] | |
63 | ||
64 | [table | |
65 | [[Expression] [Description]] | |
66 | [[`traits::is_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if | |
67 | a certain type, `L` is a Lexer, `mpl::false_` | |
68 | otherwise (See __mpl_boolean_constant__).]] | |
69 | ] | |
70 | ||
71 | [heading Postcondition] | |
72 | ||
73 | Upon return from `l.collect` the following post conditions should hold: | |
74 | ||
75 | * On return, `def` holds all token definitions defined in the Lexer, `l`. This | |
76 | includes all Lexer's contained inside `l`. | |
77 | ||
78 | Upon return from `l.add_actions` the following post conditions should hold: | |
79 | ||
80 | * On return, `def` holds all semantic actions correctly associated with the | |
81 | corresponding token definitions as defined in the Lexer, `l`. This | |
82 | includes all semantic actions defined by the Lexer's contained inside `l`. | |
83 | ||
84 | [heading Models] | |
85 | ||
86 | All lexer components in __lex__ are models of the /Lexer/ concept. | |
87 | ||
88 | [endsect] [/ Lexer Concept] | |
89 | ||
90 | [/////////////////////////////////////////////////////////////////////////////] | |
91 | [section PrimitiveLexer] | |
92 | ||
93 | [heading Description] | |
94 | ||
95 | /PrimitiveLexer/ is the most basic building block that the client uses | |
96 | to build more complex lexer components. | |
97 | ||
98 | [heading Refinement of] | |
99 | ||
100 | [:__lexer_concept__] | |
101 | ||
102 | [heading Type Expressions] | |
103 | ||
104 | [table | |
105 | [[Expression] [Description]] | |
106 | [[`traits::is_primitive_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if | |
107 | a certain type, `L`, is a PrimitiveLexer, `mpl::false_` | |
108 | otherwise (See __mpl_boolean_constant__).]] | |
109 | ] | |
110 | ||
111 | [heading Models] | |
112 | ||
113 | The following lexer components conform to this model: | |
114 | ||
115 | * character literals (i.e. `'x'`), `char_`, | |
116 | * string literals (`"abc"`), `std::basic_string<>`, `string` | |
117 | ||
118 | __fixme__ Add more links to /PrimitiveLexer/ models here. | |
119 | ||
120 | [endsect] [/ PrimitiveLexer Concept] | |
121 | ||
122 | [/////////////////////////////////////////////////////////////////////////////] | |
123 | [section UnaryLexer] | |
124 | ||
125 | [heading Description] | |
126 | ||
127 | /UnaryLexer/ is a composite lexer component that has a single subject. The | |
128 | UnaryLexer may change the behavior of its subject following the | |
129 | __delegate_pattern__. | |
130 | ||
131 | [heading Refinement of] | |
132 | ||
133 | [:__lexer_concept__] | |
134 | ||
135 | [variablelist Notation | |
136 | [[`l`] [A UnaryLexer.]] | |
137 | [[`L`] [A UnaryLexer type.]] | |
138 | ] | |
139 | ||
140 | [heading Valid Expressions] | |
141 | ||
142 | In addition to the requirements defined in __lexer_concept__, for any | |
143 | UnaryLexer the following must be met: | |
144 | ||
145 | [table | |
146 | [[Expression] [Semantics] [Return type]] | |
147 | [[`l.subject`] [Subject lexer component.] [__lexer_concept__]] | |
148 | ] | |
149 | ||
150 | [heading Type Expressions] | |
151 | ||
152 | [table | |
153 | [[Expression] [Description]] | |
154 | [[`L::subject_type`] [The subject lexer component type.]] | |
155 | [[`traits::is_unary_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if | |
156 | a certain type, `L` is a UnaryLexer, `mpl::false_` | |
157 | otherwise (See __mpl_boolean_constant__).]] | |
158 | ] | |
159 | ||
160 | [heading Invariants] | |
161 | ||
162 | For any UnaryLexer, `L`, the following invariant always holds: | |
163 | ||
164 | * `traits::is_lexer<L::subject_type>::type` evaluates to `mpl::true_` | |
165 | ||
166 | [heading Models] | |
167 | ||
168 | The following lexer components conform to this model: | |
169 | ||
170 | * action lexer component (allowing to attach semantic actions) | |
171 | ||
172 | __fixme__ Add more links to models of UnaryLexer concept | |
173 | ||
174 | [endsect] [/ UnaryLexer Concept] | |
175 | ||
176 | [/////////////////////////////////////////////////////////////////////////////] | |
177 | [section NaryLexer] | |
178 | ||
179 | [heading Description] | |
180 | ||
181 | /NaryLexer/ is a composite lexer component that has one or more subjects. The | |
182 | NaryLexer allows its subjects to be treated in the same way as a single | |
183 | instance of a __lexer_concept__ following the __composite_pattern__. | |
184 | ||
185 | [heading Refinement of] | |
186 | ||
187 | [:__lexer_concept__] | |
188 | ||
189 | [variablelist Notation | |
190 | [[`l`] [A NaryLexer.]] | |
191 | [[`L`] [A NaryLexer type.]] | |
192 | ] | |
193 | ||
194 | [heading Valid Expressions] | |
195 | ||
196 | In addition to the requirements defined in __lexer_concept__, for any | |
197 | NaryLexer the following must be met: | |
198 | ||
199 | [table | |
200 | [[Expression] [Semantics] [Return type]] | |
201 | [[`l.elements`] [The tuple of elements.] [A __fusion__ Sequence of __lexer_concept__ types.]] | |
202 | ] | |
203 | ||
204 | [heading Type Expressions] | |
205 | ||
206 | [table | |
207 | [[Expression] [Description]] | |
208 | [[`l.elements_type`] [Elements tuple type.]] | |
209 | [[`traits::is_nary_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if | |
210 | a certain type, `L` is a NaryLexer, `mpl::false_` | |
211 | otherwise (See __mpl_boolean_constant__).]] | |
212 | ] | |
213 | ||
214 | [heading Invariants] | |
215 | ||
216 | For each element, `E`, in any NaryLexer, `L`, the following | |
217 | invariant always holds: | |
218 | ||
219 | * `traits::is_lexer<E>::type` evaluates to `mpl::true_` | |
220 | ||
221 | [heading Models] | |
222 | ||
223 | The following lexer components conform to this model: | |
224 | ||
225 | * lexer sequence component | |
226 | ||
227 | __fixme__ Add more links to models of NaryLexer concept | |
228 | ||
229 | [endsect] [/ NaryLexer Concept] | |
230 | ||
231 | [endsect] |