ceph/src/boost/libs/spirit/doc/preface.qbk

   1 [/==============================================================================
   2     Copyright (C) 2001-2011 Joel de Guzman
   3     Copyright (C) 2001-2011 Hartmut Kaiser
   4
   5     Distributed under the Boost Software License, Version 1.0. (See accompanying
   6     file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
   7 ===============================================================================/]
   8
   9 [section Preface]
  10
  11 [:['["Examples of designs that meet most of the criteria for
  12 "goodness" (easy to understand, flexible, efficient) are a
  13 recursive-descent parser, which is traditional procedural
  14 code. Another example is the STL, which is a generic library of
  15 containers and algorithms depending crucially on both traditional
  16 procedural code and on parametric polymorphism.]] [*--Bjarne
  17 Stroustrup]]
  18
  19 [heading History]
  20
  21 [heading /80s/]
  22
  23 In the mid-80s, Joel wrote his first calculator in Pascal. Such an
  24 unforgettable coding experience, he was amazed at how a mutually
  25 recursive set of functions can model a grammar specification. In time,
  26 the skills he acquired from that academic experience became very
  27 practical as he was tasked to do some parsing. For instance, whenever he
  28 needed to perform any form of binary or text I/O, he tried to approach
  29 each task somewhat formally by writing a grammar using Pascal-like
  30 syntax diagrams and then a corresponding recursive-descent parser. This
  31 process worked very well.
  32
  33 [heading /90s/]
  34
  35 The arrival of the Internet and the World Wide Web magnified the need
  36 for parsing a thousand-fold. At one point Joel had to write an HTML
  37 parser for a Web browser project. Using the W3C formal specifications,
  38 he easily wrote a recursive-descent HTML parser. With the influence of
  39 the Internet, RFC specifications were abundent. SGML, HTML, XML, email
  40 addresses and even those seemingly trivial URLs were all formally
  41 specified using small EBNF-style grammar specifications. Joel had more
  42 parsing to do, and he wished for a tool similar to larger parser
  43 generators such as YACC and ANTLR, where a parser is built automatically
  44 from a grammar specification.
  45
  46 This ideal tool would be able to parse anything from email addresses and
  47 command lines, to XML and scripting languages. Scalability was a primary
  48 goal. The tool would be able to do this without incurring a heavy
  49 development load, which was not possible with the above mentioned parser
  50 generators. The result was Spirit.
  51
  52 Spirit was a personal project that was conceived when Joel was involved
  53 in R&D in Japan. Inspired by the GoF's composite and interpreter
  54 patterns, he realized that he can model a recursive-descent parser with
  55 hierarchical-object composition of primitives (terminals) and composites
  56 (productions). The original version was implemented with run-time
  57 polymorphic classes. A parser was generated at run time by feeding in
  58 production rule strings such as:
  59
  60     "prod ::= {'A' | 'B'} 'C';"
  61
  62 A compile function compiled the parser, dynamically creating a hierarchy
  63 of objects and linking semantic actions on the fly. A very early text
  64 can be found here: __early_spirit__.
  65
  66 [heading /2001 to 2006/]
  67
  68 Version 1.0 to 1.8 was a complete rewrite of the original Spirit parser
  69 using expression templates and static polymorphism, inspired by the
  70 works of Todd Veldhuizen (__exprtemplates__, C++ Report, June
  71 1995). Initially, the static-Spirit version was meant only to replace
  72 the core of the original dynamic-Spirit. Dynamic-Spirit needed a parser
  73 to implement itself anyway. The original employed a hand-coded
  74 recursive-descent parser to parse the input grammar specification
  75 strings. It was at this time when Hartmut Kaiser joined the Spirit
  76 development.
  77
  78 After its initial "open-source" debut in May 2001, static-Spirit became
  79 a success. At around November 2001, the Spirit website had an activity
  80 percentile of 98%, making it the number one parser tool at Source Forge
  81 at the time. Not bad for a niche project like a parser library. The
  82 "static" portion of Spirit was forgotten and static-Spirit simply became
  83 Spirit. The library soon evolved to acquire more dynamic features.
  84
  85 Spirit was formally accepted into __boost__ in October 2002. Boost is a
  86 peer-reviewed, open collaborative development effort around a collection
  87 of free Open Source C++ libraries covering a wide range of domains. The
  88 Boost Libraries have become widely known as an industry standard for
  89 design and implementation quality, robustness, and reusability.
  90
  91 [heading /2007/]
  92
  93 Over the years, especially after Spirit was accepted into Boost, Spirit
  94 has served its purpose quite admirably. [*/Classic-Spirit/] (versions
  95 prior to 2.0) focused on transduction parsing, where the input string is
  96 merely translated to an output string. Many parsers fall into the
  97 transduction type. When the time came to add attributes to the parser
  98 library, it was done in a rather ad-hoc manner, with the goal being 100%
  99 backward compatible with Classic Spirit. As a result, some parsers have
 100 attributes, some don't.
 101
 102 Spirit V2 is another major rewrite. Spirit V2 grammars are fully
 103 attributed (see __attr_grammar__) which means that all parser components
 104 have attributes. To do this efficiently and elegantly, we had to use a
 105 couple of infrastructure libraries. Some did not exist, some were quite
 106 new when Spirit debuted, and some needed work. __mpl__ is an important
 107 infrastructure library, yet is not sufficient to implement Spirit V2.
 108 Another library had to be written: __fusion__. Fusion sits between MPL
 109 and STL --between compile time and runtime -- mapping types to values.
 110 Fusion is a direct descendant of both MPL and __boost_tuples__. Fusion
 111 is now a full-fledged __boost__ library. __phoenix__ also had to be
 112 beefed up to support Spirit V2. The result is __boost_phoenix__. Last
 113 but not least, Spirit V2 uses an __exprtemplates__ library called
 114 __boost_proto__.
 115
 116 Even though it has evolved and matured to become a multi-module library,
 117 Spirit is still used for micro-parsing tasks as well as scripting
 118 languages. Like C++, you only pay for features that you need. The power
 119 of Spirit comes from its modularity and extensibility. Instead of giving
 120 you a sledgehammer, it gives you the right ingredients to easily create
 121 a sledgehammer.
 122
 123 [heading New Ideas: Spirit V2]
 124
 125 Just before the development of Spirit V2 began, Hartmut came across the
 126 __string_template__ library that is a part of the ANTLR parser
 127 framework. [footnote Quote from http://www.stringtemplate.org/: It is a
 128 Java template engine (with ports for C# and Python) for generating
 129 source code, web pages, emails, or any other formatted text output.]
 130 The concepts presented in that library lead Hartmut to
 131 the next step in the evolution of Spirit. Parsing and generation are
 132 tightly connected to a formal notation, or a grammar. The grammar
 133 describes both input and output, and therefore, a parser library should
 134 have a grammar driven output. This duality is expressed in Spirit by the
 135 parser library __qi__ and the generator library __karma__ using the same
 136 component infrastructure.
 137
 138 The idea of creating a lexer library well integrated with the Spirit
 139 parsers is not new. This has been discussed almost since Classic-Spirit
 140 (pre V2) initially debuted. Several attempts to integrate existing lexer
 141 libraries and frameworks with Spirit have been made and served as a
 142 proof of concept and usability (for example see __wave__: The Boost
 143 C/C++ Preprocessor Library, and __slex__: a fully dynamic C++ lexer
 144 implemented with Spirit). Based on these experiences we added __lex__: a
 145 fully integrated lexer library to the mix, allowing the user to take
 146 advantage of the power of regular expressions for token matching,
 147 removing pressure from the parser components, simplifying parser
 148 grammars. Again, Spirit's modular structure allowed us to reuse the same
 149 underlying component library as for the parser and generator libraries.
 150
 151 [heading How to use this manual]
 152
 153 Each major section (there are 3: __sec_qi__, __sec_karma__, and
 154 __sec_lex__) is roughly divided into 3 parts:
 155
 156 # Tutorials: A step by step guide with heavily annotated code. These
 157   are meant to get the user acquainted with the library as quickly as
 158   possible. The objective is to build the confidence of the user in
 159   using the library through abundant examples and detailed
 160   instructions. Examples speak volumes and we have volumes of
 161   examples!
 162
 163 # Abstracts: A high level summary of key topics. The objective is to
 164   give the user a high level view of the library, the key concepts,
 165   background and theories.
 166
 167 # Reference: Detailed formal technical reference. We start with a quick
 168   reference -- an easy to use table that maps into the reference proper.
 169   The reference proper starts with C++ concepts followed by
 170   models of the concepts.
 171
 172 Some icons are used to mark certain topics indicative of their relevance.
 173 These icons precede some text to indicate:
 174
 175 [table Icons
 176
 177     [[Icon]             [Name]          [Meaning]]
 178
 179     [[__note__]         [Note]          [Generally useful information (an aside that
 180                                         doesn't fit in the flow of the text)]]
 181
 182     [[__tip__]          [Tip]           [Suggestion on how to do something
 183                                         (especially something that is not obvious)]]
 184
 185     [[__important__]    [Important]     [Important note on something to take
 186                                         particular notice of]]
 187
 188     [[__caution__]      [Caution]       [Take special care with this - it may
 189                                         not be what you expect and may cause bad
 190                                         results]]
 191
 192     [[__danger__]       [Danger]        [This is likely to cause serious
 193                                         trouble if ignored]]
 194 ]
 195
 196 This documentation is automatically generated by Boost QuickBook
 197 documentation tool. QuickBook can be found in the __boost_tools__.
 198
 199 [heading Support]
 200
 201 Please direct all questions to Spirit's mailing list. You can subscribe
 202 to the __spirit_list__. The mailing list has a searchable archive. A
 203 search link to this archive is provided in __spirit__'s home page. You
 204 may also read and post messages to the mailing list through
 205 __spirit_general__ (thanks to __gmane__). The news group mirrors the
 206 mailing list. Here is a link to the archives: __mlist_archive__.
 207
 208 [endsect] [/ Preface]