projects / ceph.git / blame
| Commit | Line | Data |
|---|---|---|
| 7c673cae FG |
1 | <html> |
| 2 | <head> | |
| 3 | <title>In-depth The Scanner</title> | |
| 4 | <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> | |
| 5 | <link rel="stylesheet" href="theme/style.css" type="text/css"> | |
| 6 | </head> | |
| 7 | ||
| 8 | <body> | |
| 9 | <table width="100%" border="0" background="theme/bkd2.gif" cellspacing="2"> | |
| 10 | <tr> | |
| 11 | <td width="10"> | |
| 12 | </td> | |
| 13 | <td width="85%"> <font size="6" face="Verdana, Arial, Helvetica, sans-serif"><b>In-depth: | |
| 14 | The Scanner</b></font> </td> | |
| 15 | <td width="112"><a href="http://spirit.sf.net"><img src="theme/spirit.gif" width="112" height="48" align="right" border="0"></a></td> | |
| 16 | </tr> | |
| 17 | </table> | |
| 18 | <br> | |
| 19 | <table border="0"> | |
| 20 | <tr> | |
| 21 | <td width="10"></td> | |
| 22 | <td width="30"><a href="../index.html"><img src="theme/u_arr.gif" border="0"></a></td> | |
| 23 | <td width="30"><a href="indepth_the_parser.html"><img src="theme/l_arr.gif" border="0"></a></td> | |
| 24 | <td width="30"><a href="indepth_the_parser_context.html"><img src="theme/r_arr.gif" border="0"></a></td> | |
| 25 | </tr> | |
| 26 | </table> | |
| 27 | <h2>Basic Scanner API </h2> | |
| 28 | <table width="90%" border="0" align="center"> | |
| 29 | <tr> | |
| 30 | <td class="table_title" colspan="10"> class scanner </td> | |
| 31 | </tr> | |
| 32 | <tr> | |
| 33 | <tr> | |
| 34 | <td class="table_cells"><code><span class=identifier>value_t</span></code></td> | |
| 35 | <td class="table_cells">typedef: The value type of the scanner's iterator</td> | |
| 36 | </tr> | |
| 37 | <td class="table_cells"><code><span class=identifier>ref_t</span></code></td> | |
| 38 | <td class="table_cells">typedef: The reference type of the scanner's iterator</td> | |
| 39 | </tr> | |
| 40 | <td class="table_cells"><code><span class=keyword>bool </span><span class=identifier>at_end</span><span class=special>() | |
| 41 | </span><span class=keyword>const</span></code></td> | |
| 42 | <td class="table_cells">Returns true if the input is exhausted</td> | |
| 43 | </tr> | |
| 44 | <td class="table_cells"><code><span class=identifier>value_t </span><span class=keyword>operator</span><span class=special>*() | |
| 45 | </span><span class=keyword>const</span></code></td> | |
| 46 | <td class="table_cells">Dereference/get a <code><span class=identifier>value_t</span></code> | |
| 47 | from the input</td> | |
| 48 | </tr> | |
| 49 | <td class="table_cells"><code><span class=keyword> </span><span class=identifier>scanner | |
| 50 | </span><span class=keyword>const</span><span class=special>& </span><span class=keyword>operator</span><span class=special>++()</span></code></td> | |
| 51 | <td class="table_cells">move the scanner forward</td> | |
| 52 | </tr> | |
| 53 | <tr> | |
| 54 | <td class="table_cells"><code><span class=identifier>IteratorT& first</span><span class=special></span></code></td> | |
| 55 | <td class="table_cells">The iterator pointing to the current input position. | |
| 56 | Held by reference</td> | |
| 57 | </tr> | |
| 58 | <tr> | |
| 59 | <td class="table_cells"><code><span class=identifier>IteratorT </span><span class=keyword>const</span> | |
| 60 | <span class=identifier>last</span><span class=special></span></code></td> | |
| 61 | <td class="table_cells">The iterator pointing to the end of the input. Held | |
| 62 | by value</td> | |
| 63 | </tr> | |
| 64 | </table> | |
| 65 | <p> The basic behavior of the scanner is handled by policies. The actual execution | |
| 66 | of the scanner's public member functions listed in the table above is implemented | |
| 67 | by the scanner policies.</p> | |
| 68 | <p> Three sets of policies govern the behavior of the scanner. These policies | |
| 69 | make it possible to extend Spirit non-intrusively. The scanner policies allow | |
| 70 | the core-functionality to be extended without requiring any potentially destabilizing | |
| 71 | changes to the code. A library writer might provide her own policies that override | |
| 72 | the ones that are already in place to fine tune the parsing process | |
| 73 | to fit her own needs. Layers above the core might also want to take advantage | |
| 74 | of this policy based machanism. Abstract syntax tree generation, debuggers and | |
| 75 | lexers come to mind.</p> | |
| 76 | <p> There are three sets of policies that govern:</p> | |
| 77 | <ul> | |
| 78 | <li>Iteration and filtering</li> | |
| 79 | <li>Recognition and matching</li> | |
| 80 | <li>Handling semantic actions</li> | |
| 81 | </ul> | |
| 82 | <a name="iteration_policy"></a> | |
| 83 | <h2>iteration_policy</h2> | |
| 84 | <p> Here are the default policies that govern iteration and filtering:</p> | |
| 85 | <pre> | |
| 86 | <code><span class=keyword>struct </span><span class=identifier>iteration_policy | |
| 87 | </span><span class=special>{ | |
| 88 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>ScannerT</span><span class=special>> | |
| 89 | </span><span class=keyword>void | |
| 90 | </span><span class=identifier>advance</span><span class=special>(</span><span class=identifier>ScannerT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>scan</span><span class=special>) </span><span class=keyword>const | |
| 91 | </span><span class=special>{ </span><span class=special>++</span><span class=identifier>scan</span><span class=special>.</span><span class=identifier>first</span><span class=special>; </span><span class=special>} | |
| 92 | ||
| 93 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>ScannerT</span><span class=special>> | |
| 94 | </span><span class=keyword>bool </span><span class=identifier>at_end</span><span class=special>(</span><span class=identifier>ScannerT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>scan</span><span class=special>) </span><span class=keyword>const | |
| 95 | </span><span class=special>{ </span><span class=keyword>return </span><span class=identifier>scan</span><span class=special>.</span><span class=identifier>first </span><span class=special>== </span><span class=identifier>scan</span><span class=special>.</span><span class=identifier>last</span><span class=special>; </span><span class=special>} | |
| 96 | ||
| 97 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>T</span><span class=special>> | |
| 98 | </span><span class=identifier>T </span><span class=identifier>filter</span><span class=special>(</span><span class=identifier>T </span><span class=identifier>ch</span><span class=special>) </span><span class=keyword>const | |
| 99 | </span><span class=special>{ </span><span class=keyword>return </span><span class=identifier>ch</span><span class=special>; </span><span class=special>} | |
| 100 | ||
| 101 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>ScannerT</span><span class=special>> | |
| 102 | </span><span class=keyword>typename </span><span class=identifier>ScannerT</span><span class=special>::</span><span class=identifier>ref_t | |
| 103 | </span><span class=identifier>get</span><span class=special>(</span><span class=identifier>ScannerT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>scan</span><span class=special>) </span><span class=keyword>const | |
| 104 | </span><span class=special>{ </span><span class=keyword>return </span><span class=special>*</span><span class=identifier>scan</span><span class=special>.</span><span class=identifier>first</span><span class=special>; </span><span class=special>} | |
| 105 | </span><span class=special>};</span></code></pre> | |
| 106 | <table width="90%" border="0" align="center"> | |
| 107 | <tr> | |
| 108 | <td class="table_title" colspan="8"> Iteration and filtering policies </td> | |
| 109 | </tr> | |
| 110 | <tr> | |
| 111 | <tr> | |
| 112 | <td class="table_cells"><b>advance</b></td> | |
| 113 | <td class="table_cells">Move the iterator forward</td> | |
| 114 | </tr> | |
| 115 | <td class="table_cells"><b>at_end</b></td> | |
| 116 | <td class="table_cells">Return true if the input is exhausted</td> | |
| 117 | </tr> | |
| 118 | <td class="table_cells"><b>filter</b></td> | |
| 119 | <td class="table_cells">Filter a character read from the input</td> | |
| 120 | </tr> | |
| 121 | <td class="table_cells"><b>get</b></td> | |
| 122 | <td class="table_cells">Read a character from the input</td> | |
| 123 | </tr> | |
| 124 | </table> | |
| 125 | <p> The following code snippet demonstrates a simple policy that converts all | |
| 126 | characters to lower case:</p> | |
| 127 | <pre> | |
| 128 | <code><span class=keyword>struct </span><span class=identifier>inhibit_case_iteration_policy </span><span class=special>: </span><span class=keyword>public </span><span class=identifier>iteration_policy | |
| 129 | </span><span class=special>{ | |
| 130 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>CharT</span><span class=special>> | |
| 131 | </span><span class=identifier>CharT filter</span><span class=special>(</span><span class=identifier>CharT ch</span><span class=special>) </span><span class=keyword>const | |
| 132 | </span><span class=special>{ | |
| 133 | </span><span class=keyword>return </span>std::<span class=identifier>tolower</span><span class=special>(</span><span class=identifier>ch</span><span class=special>); | |
| 134 | } | |
| 135 | };</span></code></pre> | |
| 136 | <a name="match_policy"></a> | |
| 137 | <h2>match_policy</h2> | |
| 138 | <p> Here are the default policies that govern recognition and matching:</p> | |
| 139 | <pre> | |
| 140 | <code><span class=keyword>struct </span><span class=identifier>match_policy | |
| 141 | </span><span class=special>{ | |
| 142 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>T</span><span class=special>> | |
| 143 | </span><span class=keyword>struct </span><span class=identifier>result </span><span class=special> | |
| 144 | { | |
| 145 | </span><span class=keyword>typedef </span><span class=identifier>match</span><span class=special><</span><span class=identifier>T</span><span class=special>> </span><span class=identifier>type</span><span class=special>; </span><span class=special> | |
| 146 | }; | |
| 147 | ||
| 148 | </span><span class=keyword>const </span><span class=identifier>match</span><span class=special><</span><span class=identifier>nil_t</span><span class=special>> | |
| 149 | </span><span class=identifier>no_match</span><span class=special>() </span><span class=keyword>const | |
| 150 | </span><span class=special>{ </span><span class=keyword> | |
| 151 | return </span><span class=identifier>match</span><span class=special><</span><span class=identifier>nil_t</span><span class=special>>(); </span><span class=special> | |
| 152 | } | |
| 153 | ||
| 154 | </span><span class=keyword>const </span><span class=identifier>match</span><span class=special><</span><span class=identifier>nil_t</span><span class=special>> | |
| 155 | </span><span class=identifier>empty_match</span><span class=special>() </span><span class=keyword>const | |
| 156 | </span><span class=special>{ </span><span class=keyword> | |
| 157 | return </span><span class=identifier>match</span><span class=special><</span><span class=identifier>nil_t</span><span class=special>>(</span><span class=number>0</span><span class=special>, </span><span class=identifier>nil_t</span><span class=special>()); | |
| 158 | </span><span class=special>} | |
| 159 | ||
| 160 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>AttrT</span><span class=special>, </span><span class=keyword>typename </span><span class=identifier>IteratorT</span><span class=special>> | |
| 161 | </span><span class=identifier>match</span><span class=special><</span><span class=identifier>AttrT</span><span class=special>> | |
| 162 | </span><span class=identifier>create_match</span><span class=special>( | |
| 163 | </span><span class=keyword>std::size_t </span><span class=identifier>length</span><span class=special>, | |
| 164 | </span><span class=identifier>AttrT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>val</span><span class=special>, | |
| 165 | </span><span class=identifier>IteratorT </span><span class=keyword>const</span><span class=special>& </span><span class=comment>/*first*/</span><span class=special>, | |
| 166 | </span><span class=identifier>IteratorT </span><span class=keyword>const</span><span class=special>& </span><span class=comment>/*last*/</span><span class=special>) </span><span class=keyword>const | |
| 167 | </span><span class=special>{ </span><span class=keyword> | |
| 168 | return </span><span class=identifier>match</span><span class=special><</span><span class=identifier>AttrT</span><span class=special>>(</span><span class=identifier>length</span><span class=special>, </span><span class=identifier>val</span><span class=special>); </span><span class=special> | |
| 169 | } | |
| 170 | ||
| 171 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>MatchT</span><span class=special>, </span><span class=keyword>typename </span><span class=identifier>IteratorT</span><span class=special>> | |
| 172 | </span><span class=keyword>void | |
| 173 | </span><span class=identifier>group_match</span><span class=special>( | |
| 174 | </span><span class=identifier>MatchT</span><span class=special>& </span><span class=comment>/*m*/</span><span class=special>, | |
| 175 | </span><span class=identifier>parser_id </span><span class=keyword>const</span><span class=special>& </span><span class=comment>/*id*/</span><span class=special>, | |
| 176 | </span><span class=identifier>IteratorT </span><span class=keyword>const</span><span class=special>& </span><span class=comment>/*first*/</span><span class=special>, | |
| 177 | </span><span class=identifier>IteratorT </span><span class=keyword>const</span><span class=special>& </span><span class=comment>/*last*/</span><span class=special>) </span><span class=keyword>const </span><span class=special>{} | |
| 178 | ||
| 179 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>Match1T</span><span class=special>, </span><span class=keyword>typename </span><span class=identifier>Match2T</span><span class=special>> | |
| 180 | </span><span class=keyword>void | |
| 181 | </span><span class=identifier>concat_match</span><span class=special>(</span><span class=identifier>Match1T</span><span class=special>& </span><span class=identifier>l</span><span class=special>, </span><span class=identifier>Match2T </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>r</span><span class=special>) </span><span class=keyword>const | |
| 182 | </span><span class=special>{ </span><span class=identifier> | |
| 183 | l</span><span class=special>.</span><span class=identifier>concat</span><span class=special>(</span><span class=identifier>r</span><span class=special>); | |
| 184 | </span><span class=special>} | |
| 185 | </span><span class=special>};</span></code></pre> | |
| 186 | <table width="90%" border="0" align="center"> | |
| 187 | <tr> | |
| 188 | <td class="table_title" colspan="12"> Recognition and matching </td> | |
| 189 | </tr> | |
| 190 | <tr> | |
| 191 | <tr> | |
| 192 | <td class="table_cells"><b>result</b></td> | |
| 193 | <td class="table_cells">A metafunction that returns a match type given an | |
| 194 | attribute type (see In-depth: The Parser)</td> | |
| 195 | </tr> | |
| 196 | <td class="table_cells"><b>no_match</b></td> | |
| 197 | <td class="table_cells">Create a failed match</td> | |
| 198 | </tr> | |
| 199 | <td class="table_cells"><b>empty_match</b></td> | |
| 200 | <td class="table_cells">Create an empty match. An empty match is a successful | |
| 201 | epsilon match (matching length == 0)</td> | |
| 202 | </tr> | |
| 203 | <td class="table_cells"><b>create_match</b></td> | |
| 204 | <td class="table_cells">Create a match given the matching length, an attribute | |
| 205 | and the iterator pair pointing to the matching portion of the input</td> | |
| 206 | </tr> | |
| 207 | <td class="table_cells"><b>group_match</b></td> | |
| 208 | <td class="table_cells">For non terminals such as rules, this is called after | |
| 209 | a successful match has been made to allow post processing</td> | |
| 210 | </tr> | |
| 211 | <td class="table_cells"><b>concat_match</b></td> | |
| 212 | <td class="table_cells">Concatenate two match objects</td> | |
| 213 | </tr> | |
| 214 | </table> | |
| 215 | <a name="action_policy"></a> | |
| 216 | <h2>action_policy</h2> | |
| 217 | <p> The action policy has only one function for handling semantic actions:</p> | |
| 218 | <pre> | |
| 219 | <code><span class=keyword>struct </span><span class=identifier>action_policy | |
| 220 | </span><span class=special>{ | |
| 221 | </span><span class=keyword>template </span><span class=special><</span><span class=keyword>typename </span><span class=identifier>ActorT</span><span class=special>, </span><span class=keyword>typename </span><span class=identifier>AttrT</span><span class=special>, </span><span class=keyword>typename </span><span class=identifier>IteratorT</span><span class=special>> | |
| 222 | </span><span class=keyword>void | |
| 223 | </span><span class=identifier>do_action</span><span class=special>( | |
| 224 | </span><span class=identifier>ActorT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>actor</span><span class=special>, | |
| 225 | </span><span class=identifier>AttrT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>val</span><span class=special>, | |
| 226 | </span><span class=identifier>IteratorT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>first</span><span class=special>, | |
| 227 | </span><span class=identifier>IteratorT </span><span class=keyword>const</span><span class=special>& </span><span class=identifier>last</span><span class=special>) </span><span class=keyword>const</span><span class=special>; | |
| 228 | </span><span class=special>};</span></code></pre> | |
| 229 | <p> The default action policy forwards to:</p> | |
| 230 | <pre> | |
| 231 | <code><span class=identifier>actor</span><span class=special>(</span><span class=identifier>first</span><span class=special>, </span><span class=identifier>last</span><span class=special>);</span></code></pre> | |
| 232 | <p> If the attribute <tt>val</tt> is of type nil_t. Otherwise:</p> | |
| 233 | <pre> | |
| 234 | <code><span class=identifier>actor</span><span class=special>(</span><span class=identifier>val</span><span class=special>);</span></code></pre> | |
| 235 | <a name="scanner_policies_mixer"></a> | |
| 236 | <h3>scanner_policies mixer</h3> | |
| 237 | <p> The class <tt>scanner_policies</tt> combines the three scanner policy classes | |
| 238 | above into one:</p> | |
| 239 | <pre> | |
| 240 | <code><span class=keyword>template </span><span class=special>< | |
| 241 | </span><span class=keyword>typename </span><span class=identifier>IterationPolicyT </span><span class=special>= </span><span class=identifier>iteration_policy</span><span class=special>, | |
| 242 | </span><span class=keyword>typename </span><span class=identifier>MatchPolicyT </span><span class=special>= </span><span class=identifier>match_policy</span><span class=special>, | |
| 243 | </span><span class=keyword>typename </span><span class=identifier>ActionPolicyT </span><span class=special>= </span><span class=identifier>action_policy</span><span class=special>> | |
| 244 | </span><span class=keyword>struct </span><span class=identifier>scanner_policies</span><span class=special>; | |
| 245 | </span></code></pre> | |
| 246 | <p> This <i>mixer</i> class inherits from all the three policies. This scanner_policies | |
| 247 | class is then used to parameterize the scanner:</p> | |
| 248 | <pre> | |
| 249 | <code><span class=keyword>template </span><span class=special>< | |
| 250 | </span><span class=keyword>typename </span><span class=identifier>IteratorT </span><span class=special>= </span><span class=keyword>char </span><span class=keyword>const</span><span class=special>*, | |
| 251 | </span><span class=keyword>typename </span><span class=identifier>PoliciesT </span><span class=special>= </span><span class=identifier>scanner_policies</span><span class=special><> </span><span class=special>> | |
| 252 | </span><span class=keyword>class </span><span class=identifier>scanner</span><span class=special>; | |
| 253 | </span></code></pre> | |
| 254 | <p> The scanner in turn inherits from the PoliciesT.</p> | |
| 255 | <a name="rebinding_policies"></a> | |
| 256 | <h3>Rebinding Policies</h3> | |
| 257 | <p> The scanner can be made to rebind to a different set of policies anytime. | |
| 258 | It has a member function <tt>change_policies(new_policies)</tt>. Given a new | |
| 259 | set of policies, this member function creates a new scanner with the new set | |
| 260 | of policies. The result type of the <i>rebound</i> scanner can be can be obtained | |
| 261 | by calling the metafunction:</p> | |
| 262 | <pre> | |
| 263 | <code><span class=identifier>rebind_scanner_policies</span><span class=special><</span><span class=identifier>ScannerT</span><span class=special>, </span><span class=identifier>PoliciesT</span><span class=special>>::</span><span class=identifier>type</span></code></pre> | |
| 264 | <a name="rebinding_iterators"></a> | |
| 265 | <h3>Rebinding Iterators</h3> | |
| 266 | <p> The scanner can also be made to rebind to a different iterator type anytime. | |
| 267 | It has a member function <tt>change_iterator(first, last)</tt>. Given a new | |
| 268 | pair of iterator of type different from the ones held by the scanner, this member | |
| 269 | function creates a new scanner with the new pair of iterators. The result type | |
| 270 | of the <i>rebound</i> scanner can be can be obtained by calling the metafunction:</p> | |
| 271 | <pre> | |
| 272 | <code><span class=identifier>rebind_scanner_iterator</span><span class=special><</span><span class=identifier>ScannerT</span><span class=special>, </span><span class=identifier>IteratorT</span><span class=special>>::</span><span class=identifier>type</span></code></pre> | |
| 273 | <table border="0"> | |
| 274 | <tr> | |
| 275 | <td width="10"></td> | |
| 276 | <td width="30"><a href="../index.html"><img src="theme/u_arr.gif" border="0"></a></td> | |
| 277 | <td width="30"><a href="indepth_the_parser.html"><img src="theme/l_arr.gif" border="0"></a></td> | |
| 278 | <td width="30"><a href="indepth_the_parser_context.html"><img src="theme/r_arr.gif" border="0"></a></td> | |
| 279 | </tr> | |
| 280 | </table> | |
| 281 | <br> | |
| 282 | <hr size="1"> | |
| 283 | <p class="copyright">Copyright © 1998-2003 Joel de Guzman<br> | |
| 284 | <br> | |
| 285 | <font size="2">Use, modification and distribution is subject to the Boost Software | |
| 286 | License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at | |
| 287 | http://www.boost.org/LICENSE_1_0.txt)</font></p> | |
| 288 | <p class="copyright"> </p> | |
| 289 | </body> | |
| 290 | </html> |