]> git.proxmox.com Git - ceph.git/blob - ceph/src/boost/tools/build/doc/src/extending.xml
add subtree-ish sources for 12.0.3
[ceph.git] / ceph / src / boost / tools / build / doc / src / extending.xml
1 <?xml version="1.0" encoding="UTF-8"?>
2 <!DOCTYPE appendix PUBLIC "-//Boost//DTD BoostBook XML V1.0//EN"
3 "http://www.boost.org/tools/boostbook/dtd/boostbook.dtd">
4
5 <chapter id="bbv2.extender">
6 <title>Extender Manual</title>
7
8 <section id="bbv2.extender.intro">
9 <title>Introduction</title>
10
11 <para>
12 This section explains how to extend Boost.Build to accomodate your
13 local requirements&mdash;primarily to add support for non-standard
14 tools you have. Before we start, be sure you have read and understoon
15 the concept of metatarget, <xref linkend="bbv2.overview.concepts"/>,
16 which is critical to understanding the remaining material.
17 </para>
18
19 <para>
20 The current version of Boost.Build has three levels of targets, listed
21 below.
22 </para>
23
24 <variablelist>
25
26 <varlistentry>
27 <term>metatarget</term>
28 <listitem>
29 <para>
30 Object that is created from declarations in Jamfiles. May
31 be called with a set of properties to produce concrete
32 targets.
33 </para>
34 </listitem>
35 </varlistentry>
36
37 <varlistentry>
38 <term>concrete target</term>
39 <listitem>
40 <para>
41 Object that corresponds to a file or an action.
42 </para>
43 </listitem>
44 </varlistentry>
45
46 <varlistentry>
47 <term>jam target</term>
48 <listitem>
49 <para>
50 Low-level concrete target that is specific to Boost.Jam build
51 engine. Essentially a string&mdash;most often a name of file.
52 </para>
53 </listitem>
54 </varlistentry>
55
56 </variablelist>
57
58 <para>
59 In most cases, you will only have to deal with concrete targets and
60 the process that creates concrete targets from
61 metatargets. Extending metatarget level is rarely required. The jam
62 targets are typically only used inside the command line patterns.
63 </para>
64
65 <warning>
66 <para>All of the Boost.Jam target-related builtin functions, like
67 <code>DEPENDS</code> or <code>ALWAYS</code> operate on jam
68 targets. Applying them to metatargets or concrete targets has no
69 effect.</para>
70 </warning>
71
72 <section id="bbv2.extender.overview.metatargets">
73 <title>Metatargets</title>
74
75 <para>Metatarget is an object that records information specified
76 in Jamfile, such as metatarget kind, name, sources and properties,
77 and can be called with specific properties to generate concrete
78 targets. At the code level it is represented by an instance of
79 class derived from <link linkend="bbv2.reference.class.abstract-target">abstract-target</link>.
80 <footnote><para>This name is historic, and will be eventuall changed to
81 <code>metatarget</code></para></footnote>
82 </para>
83
84 <para>The <link linkend="bbv2.reference.class.abstract-target.generate">generate</link>
85 method takes the build properties
86 (as an instance of the <link linkend="bbv2.reference.class.property-set">
87 property-set</link> class) and returns
88 a list containing:</para>
89 <itemizedlist>
90 <listitem><para>As front element&mdash;Usage-requirements from this invocation
91 (an instance of <link linkend="bbv2.reference.class.property-set">
92 property-set</link>)</para></listitem>
93 <listitem><para>As subsequent elements&mdash;created concrete targets (
94 instances of the <classname>virtual-target</classname> class.)</para></listitem>
95 </itemizedlist>
96
97 <para>It's possible to lookup a metataget by target-id using the
98 <code>targets.resolve-reference</code> function, and the
99 <code>targets.generate-from-reference</code> function can both
100 lookup and generate a metatarget.</para>
101
102 <para>The <link linkend="bbv2.reference.class.abstract-target">abstract-target</link>
103 class has three immediate derived classes:</para>
104 <itemizedlist>
105
106 <listitem><para><link linkend="bbv2.reference.class.project-target">project-target</link> that
107 corresponds to a project and is not intended for further
108 subclassing. The <link linkend="bbv2.reference.class.project-target.generate">
109 generate</link> method of this
110 class builds all targets in the project that are not marked as
111 explicit.</para></listitem>
112
113 <listitem><para><link linkend="bbv2.reference.class.main-target">main-target</link>
114 corresponds to a target in a project
115 and contains one or more target alternatives. This class also should not be
116 subclassed. The <link linkend="bbv2.reference.class.main-target.generate">generate</link>
117 method of this class selects an alternative to build, and calls the
118 <link linkend="bbv2.reference.class.basic-target.generate">generate</link>
119 method of that alternative.</para></listitem>
120
121 <listitem><para><link linkend="bbv2.reference.class.basic-target">basic-target</link>
122 corresponds to a specific target alternative. This is base class,
123 with a number of derived classes. The
124 <link linkend="bbv2.reference.class.basic-target.generate">generate</link> method
125 processes the target requirements and requested build properties to
126 determine final properties for the target, builds all sources, and
127 finally calls the abstract
128 <link linkend="bbv2.reference.class.basic-target.construct">construct</link>
129 method with the list of source virtual targets, and the final properties.
130 </para></listitem>
131
132 </itemizedlist>
133
134 <para>The instances of the <link linkend="bbv2.reference.class.project-target">project-target</link> and
135 <link linkend="bbv2.reference.class.main-target">main-target</link> classes are created
136 implicitly&mdash;when loading a new Jamfiles, or when a new target
137 alternative with as-yet unknown name is created. The instances of the
138 classes derived from <link linkend="bbv2.reference.class.basic-target">basic-target</link>
139 are typically created when Jamfile calls a <firstterm>metatarget rule</firstterm>,
140 such as such as <code>exe</code>.
141 </para>
142
143 <para>It it permissible to create a custom class derived from
144 <link linkend="bbv2.reference.class.basic-target">basic-target</link> and create new metatarget rule
145 that creates instance of such target. However, in the majority
146 of cases, a specific subclass of <link linkend="bbv2.reference.class.basic-target">basic-target</link>&mdash;
147 <link linkend="bbv2.reference.class.typed-target">typed-target</link> is used. That class is associated
148 with a <firstterm>type</firstterm> and relays to <firstterm>generators</firstterm>
149 to construct concrete targets of that type. This process will be explained below.
150 When a new type is declared, a new metatarget rule is automatically defined.
151 That rule creates new instance of type-target, associated with that type.
152 </para>
153
154 </section>
155
156 <section id="bbv2.extender.overview.targets">
157 <title>Concrete targets</title>
158
159 <para>Concrete targets are represented by instance of classes derived
160 from <classname>virtual-target</classname>. The most commonly used
161 subclass is <classname>file-target</classname>. A file target is associated
162 with an action that creates it&mdash; an instance of the <classname>action</classname>
163 class. The action, in turn, hold a list of source targets. It also holds the
164 <link linkend="bbv2.reference.class.property-set">property-set</link>
165 instance with the build properties that should be used for the action.</para>
166
167 <para>Here's an example of creating a target from another target, <code>source</code></para>
168 <programlisting>
169 local a = [ new action $(source) : common.copy : $(property-set) ] ;
170 local t = [ new file-target $(name) : CPP : $(project) : $(a) ] ;
171 </programlisting>
172 <para>The first line creates an instance of the <classname>action</classname> class.
173 The first parameter is the list of sources. The second parameter is the name
174 a jam-level <link linkend="bbv2.overview.jam_language.actions">action</link>.
175 The third parameter is the property-set applying to this action. The second line
176 creates a target. We specifie a name, a type and a project. We also pass the
177 action object created earlier. If the action creates several targets, we can repeat
178 the second line several times.</para>
179
180 <para>In some cases, code that creates concrete targets may be invoked more than
181 once with the same properties. Returning to different instance of <classname>file-target</classname>
182 that correspond to the same file clearly will result in problems. Therefore, whenever
183 returning targets you should pass them via the <code>virtual-target.register</code>
184 function, besides allowing Boost.Build to track which virtual targets
185 got created for each metatarget, this will also replace targets with previously created identical
186 ones, as necessary.<footnote><para>This create-then-register pattern is caused by limitations
187 of the Boost.Jam language. Python port is likely to never create duplicate targets.</para></footnote>
188 Here are a couple of examples:
189 <programlisting>
190 return [ virtual-target.register $(t) ] ;
191 return [ sequence.transform virtual-target.register : $(targets) ] ;
192 </programlisting>
193 </para>
194
195 </section>
196
197 <section id="bbv2.extender.overview.generators">
198 <title>Generators</title>
199
200 <para>In theory, every kind of metatarget in Boost.Build (like <code>exe</code>,
201 <code>lib</code> or <code>obj</code>) could be implemented
202 by writing a new metatarget class that, independently of the other code, figures
203 what files to produce and what commands to use. However, that would be rather inflexible.
204 For example, adding support for a new compiler would require editing several metatargets.
205 </para>
206
207 <para>In practice, most files have specific types, and most tools
208 consume and produce files of specific type. To take advantage of this
209 fact, Boost.Build defines concept of target type and
210 <indexterm><primary>generators</primary></indexterm>
211 <firstterm>generators</firstterm>, and has special metatarget class
212 <link linkend="bbv2.reference.class.typed-target">typed-target</link>. Target type is merely an
213 identifier. It is associated with a set of file extensions that
214 correspond to that type. Generator is an abstraction of a tool. It advertises
215 the types it produces and, if called with a set of input target, tries to construct
216 output targets of the advertised types. Finally,
217 <link linkend="bbv2.reference.class.typed-target">typed-target</link>
218 is associated with specific target type, and relays the generator (or generators)
219 for that type.
220 </para>
221
222 <para>A generator is an instance of a class derived from <classname>generator</classname>.
223 The <classname>generator</classname> class itself is suitable for common cases.
224 You can define derived classes for custom scenarios.</para>
225
226 <!--
227 <para>Given a set of generators, the fundamental operation is to
228 construct a target of a given type, with given properties, from a
229 set of targets. That operation is performed by rule
230 <literal>generators.construct</literal> and the used algorithm is described
231 below.</para>
232
233 <section>
234 <title>Selecting and ranking viable generators</title>
235
236 <para>Each generator, in addition to target types that it can
237 produce, have attribute that affects its applicability in
238 particular sitiation. Those attributes are:</para>
239
240 <orderedlist>
241 <listitem>
242 <simpara>
243 Required properties, which are properties absolutely
244 necessary for the generator to work. For example, generator
245 encapsulating the gcc compiler would have &lt;toolset&gt;gcc as
246 required property.
247 </simpara>
248 </listitem>
249
250 <listitem>
251 <simpara>
252 Optional properties, which increase the generators
253 suitability for a particual build.
254 </simpara>
255 </listitem>
256 </orderedlist>
257
258 <para>
259 Generator's required and optional properties may not include
260 either free or incidental properties. (Allowing this would
261 greatly complicate caching targets).
262 </para>
263
264 <para>When trying to construct a target, the first step is to select
265 all possible generators for the requested target type, which
266 required properties are a subset of requested properties.
267 Generators that were already selected up the call stack are
268 excluded. In addition, if any composing generators were selected
269 up the call stack, all other composing generators are ignored
270 (TODO: define composing generators). The found generators
271 are assigned a rank, which is the number of optional properties
272 present in requested properties. Finally, generators with highest
273 rank are selected for futher processing.</para>
274
275 </section>
276 <section>
277 <title>Running generators</title>
278
279 <para>When generators are selected, each is run to produce a list of
280 created targets. This list might include targets that are not of
281 requested types, because generators create the same targets as
282 some tool, and tool's behaviour is fixed. (Note: should specify
283 that in some cases we actually want extra targets). If generator
284 fails, it returns an empty list. Generator is free to call
285 'construct' again, to convert sources to the types it can handle.
286 It also can pass modified properties to 'construct'. However, a
287 generator is not allowed to modify any propagated properties,
288 otherwise when actually consuming properties we might discover
289 that the set of propagated properties is different from what was
290 used for building sources.</para>
291
292 <para>For all targets that are not of requested types, we try to
293 convert them to requested type, using a second call to
294 <literal>construct</literal>. This is done in order to support
295 transformation sequences where single source file expands to
296 several later. See <ulink url=
297 "http://groups.yahoo.com/group/jamboost/message/1667">this
298 message</ulink> for details.</para>
299
300 </section>
301
302 -->
303
304 <!-- FIXME: review the below content. Maybe, some of it is
305 still useful.
306 <section>
307 <title>Property adjustment</title>
308
309 <para>Because target location is determined by the build system, it
310 is sometimes necessary to adjust properties, in order to not
311 break actions. For example, if there's an action that generates
312 a header, say "a_parser.h", and a source file "a.cpp" which
313 includes that file, we must make everything work as if a_parser.h
314 is generated in the same directory where it would be generated
315 without any subvariants.</para>
316
317 <para>Correct property adjustment can be done only after all targets
318 are created, so the approach taken is:</para>
319
320 <orderedlist>
321 <listitem>
322 <para>
323 When dependency graph is constructed, each action can be
324 assigned a rule for property adjustment.
325 </para>
326 </listitem>
327
328 <listitem>
329 <para>
330 When virtual target is actualized, that rule is run and
331 return the final set of properties. At this stage it can use
332 information of all created virtual targets.
333 </para>
334 </listitem>
335 </orderedlist>
336
337 <para>In case of quoted includes, no adjustment can give 100% correct
338 results. If target dirs are not changed by build system, quoted
339 includes are searched in "." and then in include path, while angle
340 includes are searched only in include path. When target dirs are
341 changed, we'd want to make quoted includes to be search in "." then in
342 additional dirs and then in the include path and make angle includes
343 be searched in include path, probably with additional paths added at
344 some position. Unless, include path already has "." as the first
345 element, this is not possible. So, either generated headers should not
346 be included with quotes, or first element of include path should be
347 ".", which essentially erases the difference between quoted and angle
348 includes. <emphasis role="bold">Note:</emphasis> the only way to get
349 "." as include path into compiler command line is via verbatim
350 compiler option. In all other case, Boost.Build will convert "." into
351 directory where it occurs.</para>
352
353 </section>
354
355 -->
356
357 </section>
358
359 </section>
360
361 <section id="bbv2.extender.example">
362 <title>Example: 1-to-1 generator</title>
363
364 <para>Say you're writing an application that generates C++ code. If
365 you ever did this, you know that it's not nice. Embedding large
366 portions of C++ code in string literals is very awkward. A much
367 better solution is:</para>
368
369 <orderedlist>
370 <listitem>
371 <simpara>
372 Write the template of the code to be generated, leaving
373 placeholders at the points that will change
374 </simpara>
375 </listitem>
376
377 <listitem>
378 <simpara>
379 Access the template in your application and replace
380 placeholders with appropriate text.
381 </simpara>
382 </listitem>
383
384 <listitem>
385 <simpara>Write the result.</simpara>
386 </listitem>
387 </orderedlist>
388
389 <para>It's quite easy to achieve. You write special verbatim files that are
390 just C++, except that the very first line of the file contains the name of a
391 variable that should be generated. A simple tool is created that takes a
392 verbatim file and creates a cpp file with a single <code>char*</code> variable
393 whose name is taken from the first line of the verbatim file and whose value
394 is the file's properly quoted content.</para>
395
396 <para>Let's see what Boost.Build can do.</para>
397
398 <para>First off, Boost.Build has no idea about "verbatim files". So, you must
399 register a new target type. The following code does it:</para>
400
401 <programlisting>
402 import type ;
403 type.register VERBATIM : verbatim ;
404 </programlisting>
405
406 <para>The first parameter to <link linkend="bbv2.reference.modules.type.register">type.register</link> gives
407 the name of the declared type. By convention, it's uppercase. The second
408 parameter is the suffix for files of this type. So, if Boost.Build sees
409 <filename>code.verbatim</filename> in a list of sources, it knows that it's of
410 type <code>VERBATIM</code>.</para>
411
412 <para>Next, you tell Boost.Build that the verbatim files can be
413 transformed into C++ files in one build step. A
414 <firstterm>generator</firstterm> is a template for a build step that
415 transforms targets of one type (or set of types) into another. Our
416 generator will be called <code>verbatim.inline-file</code>; it
417 transforms <code>VERBATIM</code> files into <code>CPP</code> files:
418
419 <programlisting>
420 import generators ;
421 generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
422 </programlisting>
423 </para>
424
425 <para>Lastly, you have to inform Boost.Build about the shell
426 commands used to make that transformation. That's done with an
427 <code>actions</code> declaration.
428
429 <programlisting>
430 actions inline-file
431 {
432 "./inline-file.py" $(&lt;) $(&gt;)
433 }
434 </programlisting>
435
436 <!-- You need to explain all the parameters to an "actions" and
437 describe the accompanying rule declaration: the user has no clue
438 what $(<) and $(>) are, and doesn't know about the third
439 parameter that gets passed to the rule. -->
440
441 <!-- We use verbatim.inline-file in one place and just inline-file in
442 another. Is this confusing for user?
443 -->
444 </para>
445
446 <para>
447 Now, we're ready to tie it all together. Put all the code above in file
448 <filename>verbatim.jam</filename>, add <code>import verbatim ;</code> to
449 <filename>Jamroot.jam</filename>, and it's possible to write the following
450 in your Jamfile:
451 </para>
452
453 <programlisting>
454 exe codegen : codegen.cpp class_template.verbatim usage.verbatim ;
455 </programlisting>
456
457 <para>
458 The listed verbatim files will be automatically converted into C++ source
459 files, compiled and then linked to the codegen executable.
460 </para>
461
462 <para>
463 In subsequent sections, we will extend this example, and review all the
464 mechanisms in detail. The complete code is available in the
465 <filename>example/customization</filename> directory.
466 </para>
467 </section>
468
469 <section id="bbv2.extending.targets">
470 <title>Target types</title>
471 <para>The first thing we did in the <link
472 linkend="bbv2.extender.intro">introduction</link> was declaring a
473 new target type:
474 <programlisting>
475 import type ;
476 type.register VERBATIM : verbatim ;
477 </programlisting>
478 The type is the most important property of a target. Boost.Build can
479 automatically generate necessary build actions only because you
480 specify the desired type (using the different main target rules), and
481 because Boost.Build can guess the type of sources from their
482 extensions.
483 </para>
484
485 <para>The first two parameters for the <code>type.register</code> rule
486 are the name of new type and the list of extensions associated with
487 it. A file with an extension from the list will have the given target
488 type. In the case where a target of the declared type is generated
489 from other sources, the first specified extension will be used.
490 </para>
491
492 <para>Sometimes you want to change the suffix used for generated targets
493 depending on build properties, such as toolset. For example, some compiler
494 uses extension <literal>elf</literal> for executable files. You can use the
495 <code>type.set-generated-target-suffix</code> rule:
496 <programlisting>
497 type.set-generated-target-suffix EXE : &lt;toolset&gt;elf : elf ;
498 </programlisting>
499 </para>
500
501 <para>A new target type can be inherited from an existing one.
502 <programlisting>
503 type.register PLUGIN : : SHARED_LIB ;
504 </programlisting>
505 The above code defines a new type derived from
506 <code>SHARED_LIB</code>. Initially, the new type inherits all the
507 properties of the base type - in particular generators and suffix.
508 Typically, you'll change the new type in some way. For example, using
509 <code>type.set-generated-target-suffix</code> you can set the suffix for
510 the new type. Or you can write special a generator for the new type. For
511 example, it can generate additional metainformation for the plugin.
512 In either way, the <code>PLUGIN</code> type can be used whenever
513 <code>SHARED_LIB</code> can. For example, you can directly link plugins
514 to an application.
515 </para>
516
517 <para>A type can be defined as "main", in which case Boost.Build will
518 automatically declare a main target rule for building targets of that
519 type. More details can be found <link
520 linkend="bbv2.extending.rules.main-type">later</link>.
521 </para>
522
523 <section id="bbv2.extending.scanners">
524 <title>Scanners</title>
525 <para>
526 Sometimes, a file can refer to other files via some include system. To
527 make Boost.Build track dependencies between included files, you need
528 to provide a scanner. The primary limitation is that only one scanner
529 can be assigned to a target type.
530 </para>
531
532 <para>First, we need to declare a new class for the scanner:
533 <programlisting>
534 class verbatim-scanner : common-scanner
535 {
536 rule pattern ( )
537 {
538 return "//###include[ ]*\"([^\"]*)\"" ;
539 }
540 }
541 </programlisting>
542 All the complex logic is in the <code>common-scanner</code>
543 class, and you only need to override the method that returns
544 the regular expression to be used for scanning. The
545 parentheses in the regular expression indicate which part
546 of the string is the name of the included file. Only the
547 first parenthesized group in the regular expression will be
548 recognized; if you can't express everything you want that
549 way, you can return multiple regular expressions, each of
550 which contains a parenthesized group to be matched.
551 </para>
552
553 <para>After that, we need to register our scanner class:
554 <programlisting>
555 scanner.register verbatim-scanner : include ;
556 </programlisting>
557 The value of the second parameter, in this case
558 <code>include</code>, specifies the properties that contain the list
559 of paths that should be searched for the included files.
560 </para>
561
562 <para>Finally, we assign the new scanner to the <code>VERBATIM</code>
563 target type:
564 <programlisting>
565 type.set-scanner VERBATIM : verbatim-scanner ;
566 </programlisting>
567 That's enough for scanning include dependencies.
568 </para>
569
570 </section>
571
572 </section>
573
574 <section id="bbv2.extending.tools">
575 <title>Tools and generators</title>
576 <para>
577 This section will describe how Boost.Build can be extended to support
578 new tools.
579 </para>
580
581 <para>For each additional tool, a Boost.Build object called generator
582 must be created. That object has specific types of targets that it
583 accepts and produces. Using that information, Boost.Build is able
584 to automatically invoke the generator. For example, if you declare a
585 generator that takes a target of the type <literal>D</literal> and
586 produces a target of the type <literal>OBJ</literal>, when placing a
587 file with extention <literal>.d</literal> in a list of sources will
588 cause Boost.Build to invoke your generator, and then to link the
589 resulting object file into an application. (Of course, this requires
590 that you specify that the <literal>.d</literal> extension corresponds
591 to the <literal>D</literal> type.)
592 </para>
593
594 <para>Each generator should be an instance of a class derived from the
595 <code>generator</code> class. In the simplest case, you don't need to
596 create a derived class, but simply create an instance of the
597 <code>generator</code> class. Let's review the example we've seen in the
598 <link linkend="bbv2.extender.intro">introduction</link>.
599 <!-- Is the following supposed to be verbatim.jam? Tell the
600 user so. You also need to describe the meanings of $(<)
601 and $(>); this is the first time they're encountered. -->
602 <programlisting>
603 import generators ;
604 generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
605 actions inline-file
606 {
607 "./inline-file.py" $(&lt;) $(&gt;)
608 }
609 </programlisting>
610 </para>
611
612 <para>We declare a standard generator, specifying its id, the source type
613 and the target type. When invoked, the generator will create a target
614 of type <literal>CPP</literal> with a source target of
615 type <literal>VERBATIM</literal> as the only source. But what command
616 will be used to actually generate the file? In Boost.Build, actions are
617 specified using named "actions" blocks and the name of the action
618 block should be specified when creating targets. By convention,
619 generators use the same name of the action block as their own id. So,
620 in above example, the "inline-file" actions block will be used to
621 convert the source into the target.
622 </para>
623
624 <para>
625 There are two primary kinds of generators: standard and composing,
626 which are registered with the
627 <code>generators.register-standard</code> and the
628 <code>generators.register-composing</code> rules, respectively. For
629 example:
630 <programlisting>
631 generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
632 generators.register-composing mex.mex : CPP LIB : MEX ;
633 </programlisting>
634 The first (standard) generator takes a <emphasis>single</emphasis>
635 source of type <code>VERBATIM</code> and produces a result. The second
636 (composing) generator takes any number of sources, which can have either
637 the <code>CPP</code> or the <code>LIB</code> type. Composing generators
638 are typically used for generating top-level target type. For example,
639 the first generator invoked when building an <code>exe</code> target is
640 a composing generator corresponding to the proper linker.
641 </para>
642
643 <para>You should also know about two specific functions for registering
644 generators: <code>generators.register-c-compiler</code> and
645 <code>generators.register-linker</code>. The first sets up header
646 dependecy scanning for C files, and the seconds handles various
647 complexities like searched libraries. For that reason, you should always
648 use those functions when adding support for compilers and linkers.
649 </para>
650
651 <para>(Need a note about UNIX)</para>
652 <!-- What kind of note? Either write the note or don't, but remove this dross. -->
653 <bridgehead>Custom generator classes</bridgehead>
654
655 <para>The standard generators allows you to specify source and target
656 types, an action, and a set of flags. If you need anything more complex,
657 <!-- What sort of flags? Command-line flags? What does the system do with them? -->
658 you need to create a new generator class with your own logic. Then,
659 you have to create an instance of that class and register it. Here's
660 an example how you can create your own generator class:
661 <programlisting>
662 class custom-generator : generator
663 {
664 rule __init__ ( * : * )
665 {
666 generator.__init__ $(1) : $(2) : $(3) : $(4) : $(5) : $(6) : $(7) : $(8) : $(9) ;
667 }
668 <!-- What is the point of this __init__ function?? -->
669 }
670
671 generators.register
672 [ new custom-generator verbatim.inline-file : VERBATIM : CPP ] ;
673 </programlisting>
674 This generator will work exactly like the
675 <code>verbatim.inline-file</code> generator we've defined above, but
676 it's possible to customize the behaviour by overriding methods of the
677 <code>generator</code> class.
678 </para>
679
680 <para>There are two methods of interest. The <code>run</code> method is
681 responsible for the overall process - it takes a number of source targets,
682 converts them to the right types, and creates the result. The
683 <code>generated-targets</code> method is called when all sources are
684 converted to the right types to actually create the result.
685 </para>
686
687 <para>The <code>generated-targets</code> method can be overridden when you
688 want to add additional properties to the generated targets or use
689 additional sources. For a real-life example, suppose you have a program
690 analysis tool that should be given a name of executable and the list of
691 all sources. Naturally, you don't want to list all source files
692 manually. Here's how the <code>generated-targets</code> method can find
693 the list of sources automatically:
694 <programlisting>
695 class itrace-generator : generator {
696 ....
697 rule generated-targets ( sources + : property-set : project name ? )
698 {
699 local leaves ;
700 local temp = [ virtual-target.traverse $(sources[1]) : : include-sources ] ;<!-- You must explain include-sources! -->
701 for local t in $(temp)
702 {
703 if ! [ $(t).action<!-- In what namespace is this evaluated? --> ]
704 {
705 leaves += $(t) ;
706 }
707 }
708 return [ generator.generated-targets $(sources) $(leafs)
709 : $(property-set) : $(project) $(name) ] ;
710 }
711 }
712 generators.register [ new itrace-generator nm.itrace : EXE : ITRACE ] ;
713 </programlisting>
714 The <code>generated-targets</code> method will be called with a single
715 source target of type <literal>EXE</literal>. The call to
716 <code>virtual-target.traverse</code> will return all targets the
717 executable depends on, and we further find files that are not
718 produced from anything. <!-- What does "not produced from anything" mean? -->
719 The found targets are added to the sources.
720 </para>
721
722 <para>The <code>run</code> method can be overriden to completely
723 customize the way the generator works. In particular, the conversion of
724 sources to the desired types can be completely customized. Here's
725 another real example. Tests for the Boost Python library usually
726 consist of two parts: a Python program and a C++ file. The C++ file is
727 compiled to Python extension that is loaded by the Python
728 program. But in the likely case that both files have the same name,
729 the created Python extension must be renamed. Otherwise, the Python
730 program will import itself, not the extension. Here's how it can be
731 done:
732 <programlisting>
733 rule run ( project name ? : property-set : sources * )
734 {
735 local python ;
736 for local s in $(sources)
737 {
738 if [ $(s).type ] = PY
739 {
740 python = $(s) ;
741 }
742 }
743 <!-- This is horrible code. Use a filter function, or at _least_ consolidate the two loops! -->
744 local libs ;
745 for local s in $(sources)
746 {
747 if [ type.is-derived [ $(s).type ] LIB ]
748 {
749 libs += $(s) ;
750 }
751 }
752
753 local new-sources ;
754 for local s in $(sources)
755 {
756 if [ type.is-derived [ $(s).type ] CPP ]
757 {
758 local name = [ $(s).name ] ; # get the target's basename
759 if $(name) = [ $(python).name ]
760 {
761 name = $(name)_ext ; # rename the target
762 }
763 new-sources += [ generators.construct $(project) $(name) :
764 PYTHON_EXTENSION : $(property-set) : $(s) $(libs) ] ;
765 }
766 }
767
768 result = [ construct-result $(python) $(new-sources) : $(project) $(name)
769 : $(property-set) ] ;
770 }
771 </programlisting>
772 <!-- Why are we doing this with a generator??? It seems
773 insane. We could just use a nice front-end rule that
774 calls some normal target-creation rules. No? -->
775
776 First, we separate all source into python files, libraries and C++
777 sources. For each C++ source we create a separate Python extension by
778 calling <code>generators.construct</code> and passing the C++ source
779 and the libraries. At this point, we also change the extension's name,
780 if necessary.
781 </para>
782
783
784 </section>
785
786 <section id="bbv2.extending.features">
787 <title>Features</title>
788 <para>
789 Often, we need to control the options passed the invoked tools. This
790 is done with features. Consider an example:
791 <programlisting>
792 # Declare a new free feature
793 import feature : feature ;
794 feature verbatim-options : : free ;
795
796 # Cause the value of the 'verbatim-options' feature to be
797 # available as 'OPTIONS' variable inside verbatim.inline-file
798 import toolset : flags ;
799 flags verbatim.inline-file OPTIONS &lt;verbatim-options&gt; ;<!-- You must tell the reader what the syntax of the flags rule is -->
800
801 # Use the "OPTIONS" variable
802 actions inline-file
803 {
804 "./inline-file.py" $(OPTIONS) $(&lt;) $(&gt;)
805 }
806 </programlisting>
807 We first define a new feature. Then, the <code>flags</code> invocation
808 says that whenever verbatin.inline-file action is run, the value of
809 the <code>verbatim-options</code> feature will be added to the
810 <code>OPTIONS</code> variable, and can be used inside the action body.
811 You'd need to consult online help (--help) to find all the features of
812 the <code>toolset.flags</code> rule.
813 <!-- It's been a while since I wrote these notes, so I don't
814 remember what I meant. But right here, I wrote "bad" and
815 circled it. Maybe you can figure out what I meant. ;-)
816 -->
817 </para>
818
819 <para>
820 Although you can define any set of features and interpret their values
821 in any way, Boost.Build suggests the following coding standard for
822 designing features.
823 </para>
824
825 <para>Most features should have a fixed set of values that is portable
826 (tool neutral) across the class of tools they are designed to work
827 with. The user does not have to adjust the values for a exact tool. For
828 example, <code>&lt;optimization&gt;speed</code> has the same meaning for
829 all C++ compilers and the user does not have to worry about the exact
830 options passed to the compiler's command line.
831 </para>
832
833 <para>
834 Besides such portable features there are special 'raw' features that
835 allow the user to pass any value to the command line parameters for a
836 particular tool, if so desired. For example, the
837 <code>&lt;cxxflags&gt;</code> feature allows you to pass any command line
838 options to a C++ compiler. The <code>&lt;include&gt;</code> feature
839 allows you to pass any string preceded by <code>-I</code> and the interpretation
840 is tool-specific. <!-- It's really tool-specific? That surprises me --> (See <xref
841 linkend="bbv2.faq.external"/> for an example of very smart usage of that
842 feature). Of course one should always strive to use portable
843 features, but these are still be provided as a backdoor just to make
844 sure Boost.Build does not take away any control from the user.
845 </para>
846
847 <para>
848 Using portable features is a good idea because:
849 <itemizedlist>
850 <listitem>
851 <para>When a portable feature is given a fixed set of
852 values, you can build your project with two different
853 settings of the feature and Boost.Build will automatically
854 use two different directories for generated files.
855 Boost.Build does not try to separate targets built with
856 different raw options.
857 <!-- It's a computer program. It doesn't "care" about options -->
858 </para>
859 </listitem>
860
861 <listitem>
862 <para>Unlike with “raw” features, you don't need to use
863 specific command-line flags in your Jamfile, and it will be
864 more likely to work with other tools.
865 </para>
866 </listitem>
867 </itemizedlist>
868 </para>
869
870 <bridgehead>Steps for adding a feauture</bridgehead>
871 <!-- This section is redundant with the previous one -->
872 <para>Adding a feature requires three steps:
873
874 <orderedlist>
875 <listitem><para>Declaring a feature. For that, the "feature.feature"
876 rule is used. You have to decide on the set of <link
877 linkend="bbv2.reference.features.attributes">feature
878 attributes</link>:
879
880 <itemizedlist>
881 <listitem><para>if you want a feature value set for one target
882 to automaticaly propagate to its dependant targets then make it
883 “propagated”. <!-- Examples needed. --></para></listitem>
884
885 <listitem><para>if a feature does not have a fixed list of
886 values, it must be “free.” For example, the <code>include
887 </code> feature is a free feature.</para></listitem>
888
889 <listitem><para>if a feature is used to refer to a path relative
890 to the Jamfile, it must be a “path” feature. Such features will
891 also get their values automatically converted to Boost.Build's
892 internal path representation. For example, <code>include</code>
893 is a path feature.</para></listitem>
894
895 <listitem><para>if feature is used to refer to some target, it
896 must be a “dependency” feature. <!-- for example? --></para>
897
898 <!-- Any other feature attributes? -->
899 </listitem>
900 </itemizedlist>
901 </para>
902 </listitem>
903
904 <listitem><para>Representing the feature value in a
905 target-specific variable. Build actions are command
906 templates modified by Boost.Jam variable expansions. The
907 <code>toolset.flags</code> rule sets a target-specific
908 variable to the value of a feature.</para></listitem>
909
910 <listitem><para>Using the variable. The variable set in step 2 can
911 be used in a build action to form command parameters or
912 files.</para></listitem>
913
914 </orderedlist>
915 </para>
916
917 <bridgehead>Another example</bridgehead>
918
919 <para>Here's another example.
920 Let's see how we can make a feature that refers to a target. For example,
921 when linking dynamic libraries on Windows, one sometimes needs to
922 specify a "DEF file", telling what functions should be exported. It
923 would be nice to use this file like this:
924 <programlisting>
925 lib a : a.cpp : &lt;def-file&gt;a.def ;
926 </programlisting>
927 <!-- Why would that be nice? It seems to me that having a.def in the sources is the obvious and much nicer thing to do:
928
929 lib a : a.cpp a.def ;
930 -->
931 Actually, this feature is already supported, but anyway...
932 <!-- Something about saying that is very off-putting. I'm
933 sorry that I can't put my finger on it -->
934 </para>
935
936 <orderedlist>
937 <listitem>
938 <para>Since the feature refers to a target, it must be "dependency".
939 <programlisting>
940 feature def-file : : free dependency ;
941 </programlisting>
942 </para></listitem>
943
944 <listitem><para>One of the toolsets that cares about
945 <!-- The toolset doesn't "care." What do your really mean? -->
946 DEF files is msvc. The following line should be added to it.
947 <!-- Are you saying the msvc toolset is broken (or that it
948 doesn't use DEF files) as-shipped and the reader needs to
949 fix it? -->
950
951 <programlisting>
952 flags msvc.link DEF_FILE &lt;def-file&gt; ;
953 </programlisting>
954 <!-- And that line does... what? -->
955 </para></listitem>
956
957 <listitem><para>Since the DEF_FILE variable is not used by the
958 msvc.link action,
959 <!-- It's not? You just told us that MSVC "cares" about DEF files. I
960 presume that means that it uses them in some appropriate way? -->
961 we need to modify it to be:
962
963 <programlisting>
964 actions link bind DEF_FILE
965 {
966 $(.LD) .... /DEF:$(DEF_FILE) ....
967 }
968 </programlisting>
969 </para>
970
971
972 <para> Note the <code>bind DEF_FILE</code> part. It tells
973 Boost.Build to translate the internal target name in
974 <varname>DEF_FILE</varname> to a corresponding filename in
975 the <code>link</code> action. Without it the expansion of
976 <code>$(DEF_FILE)</code> would be a strange symbol that is
977 not likely to make sense for the linker.
978 </para>
979
980 <!-- I have a note here that says: "none of this works for
981 targets in general, only source files." I'm not sure
982 what I meant by that; maybe you can figure it out. -->
983 <para>
984 We are almost done, except for adding the follwing code to <filename>msvc.jam</filename>:
985
986 <programlisting>
987 rule link
988 {
989 DEPENDS $(&lt;) : [ on $(&lt;) return $(DEF_FILE) ] ;
990 }
991 </programlisting>
992 <!-- You *must* explain the part in [...] above. It's completely opaque to the casual reader -->
993
994 This is a workaround for a bug in Boost.Build engine, which will hopefully
995 be fixed one day.
996 <!-- This is *NOT* a bug!! Anyway, BBv2 shouild handle this automatically. Why doesn't it? -->
997 </para></listitem>
998
999 </orderedlist>
1000
1001 <bridgehead>Variants and composite features.</bridgehead>
1002
1003 <para>Sometimes you want to create a shortcut for some set of
1004 features. For example, <code>release</code> is a value of
1005 <code>&lt;variant&gt;</code> and is a shortcut for a set of features.
1006 </para>
1007
1008 <para>It is possible to define your own build variants. For example:
1009 <programlisting>
1010 variant crazy : &lt;optimization&gt;speed &lt;inlining&gt;off
1011 &lt;debug-symbols&gt;on &lt;profiling&gt;on ;
1012 </programlisting>
1013 will define a new variant with the specified set of properties. You
1014 can also extend an existing variant:
1015 <programlisting>
1016 variant super_release : release : &lt;define&gt;USE_ASM ;
1017 </programlisting>
1018 In this case, <code>super_release</code> will expand to all properties
1019 specified by <code>release</code>, and the additional one you've specified.
1020 </para>
1021
1022 <para>You are not restricted to using the <code>variant</code> feature
1023 only.
1024 <!-- What do you mean by that? How is defining a new feature related to what came before? -->
1025 Here's example that defines a brand new feature:
1026 <programlisting>
1027 feature parallelism : mpi fake none : composite link-incompatible ;
1028 feature.compose &lt;parallelism&gt;mpi : &lt;library&gt;/mpi//mpi/&lt;parallelism&gt;none ;
1029 feature.compose &lt;parallelism&gt;fake : &lt;library&gt;/mpi//fake/&lt;parallelism&gt;none ;
1030 </programlisting>
1031 <!-- The use of the <library>/mpi//mpi/<parallelism>none construct
1032 above is at best confusing and unexplained -->
1033 This will allow you to specify the value of feature
1034 <code>parallelism</code>, which will expand to link to the necessary
1035 library.
1036 </para>
1037
1038 </section>
1039
1040 <section id="bbv2.extending.rules">
1041 <title>Main target rules</title>
1042 <para>
1043 A main target rule (e.g “<link linkend="bbv2.tasks.programs">exe</link>”
1044 Or “<link linkend="bbv2.tasks.libraries">lib</link>”) creates a top-level target. It's quite likely that you'll want to declare your own and
1045 there are two ways to do that.
1046 <!-- Why did "that" get changed to "this" above? -->
1047 </para>
1048
1049 <para id="bbv2.extending.rules.main-type">The first way applies when
1050 <!-- This is not a "way of defining a main target rule." Rephrase this and the previous sentence. -->
1051 your target rule should just produce a target of specific type. In that case, a
1052 rule is already defined for you! When you define a new type, Boost.Build
1053 automatically defines a corresponding rule. The name of the rule is
1054 obtained from the name of the type, by downcasing all letters and
1055 replacing underscores with dashes.
1056 <!-- This strikes me as needless complexity, and confusing. Why
1057 do we have the uppercase-underscore convention for target
1058 types? If we just dropped that, the rule names could be
1059 the same as the type names. -->
1060 For example, if you create a module
1061 <filename>obfuscate.jam</filename> containing:
1062
1063 <programlisting>
1064 import type ;
1065 type.register OBFUSCATED_CPP : ocpp ;
1066
1067 import generators ;
1068 generators.register-standard obfuscate.file : CPP : OBFUSCATED_CPP ;
1069 </programlisting>
1070 and import that module, you'll be able to use the rule "obfuscated-cpp"
1071 in Jamfiles, which will convert source to the OBFUSCATED_CPP type.
1072 </para>
1073
1074 <para>
1075 The second way is to write a wrapper rule that calls any of the existing
1076 rules. For example, suppose you have only one library per directory and
1077 want all cpp files in the directory to be compiled into that library. You
1078 can achieve this effect using:
1079 <programlisting>
1080 lib codegen : [ glob *.cpp ] ;
1081 </programlisting>
1082 If you want to make it even simpler, you could add the following
1083 definition to the <filename>Jamroot.jam</filename> file:
1084 <programlisting>
1085 rule glib ( name : extra-sources * : requirements * )
1086 {
1087 lib $(name) : [ glob *.cpp ] $(extra-sources) : $(requirements) ;
1088 }
1089 </programlisting>
1090 allowing you to reduce the Jamfile to just
1091 <programlisting>
1092 glib codegen ;
1093 </programlisting>
1094 </para>
1095
1096 <para>
1097 Note that because you can associate a custom generator with a target type,
1098 the logic of building can be rather complicated. For example, the
1099 <code>boostbook</code> module declares a target type
1100 <code>BOOSTBOOK_MAIN</code> and a custom generator for that type. You can
1101 use that as example if your main target rule is non-trivial.
1102 </para>
1103 </section>
1104
1105 <section id="bbv2.extending.toolset_modules">
1106
1107 <title>Toolset modules</title>
1108
1109 <para>
1110 If your extensions will be used only on one project, they can be placed in
1111 a separate <filename>.jam</filename> file and imported by your
1112 <filename>Jamroot.jam</filename>. If the extensions will be used on many
1113 projects, users will thank you for a finishing touch.
1114 </para>
1115
1116 <para>The <code>using</code> rule provides a standard mechanism
1117 for loading and configuring extensions. To make it work, your module
1118 <!-- "module" hasn't been defined yet. Furthermore you haven't
1119 said anything about where that module file must be
1120 placed. -->
1121 should provide an <code>init</code> rule. The rule will be called
1122 with the same parameters that were passed to the
1123 <code>using</code> rule. The set of allowed parameters is
1124 determined by you. For example, you can allow the user to specify
1125 paths, tool versions, and other options.
1126 <!-- But it's not entirely arbitrary. We have a standard
1127 parameter order which you should describe here for
1128 context. -->
1129 </para>
1130
1131 <para>Here are some guidelines that help to make Boost.Build more
1132 consistent:
1133 <itemizedlist>
1134 <listitem><para>The <code>init</code> rule should never fail. Even if
1135 the user provided an incorrect path, you should emit a warning and go
1136 on. Configuration may be shared between different machines, and
1137 wrong values on one machine can be OK on another.
1138 <!-- So why shouldn't init fail on machines where it's wrong?? -->
1139 </para></listitem>
1140
1141 <listitem><para>Prefer specifying the command to be executed
1142 to specifying the tool's installation path. First of all, this
1143 gives more control: it's possible to specify
1144 <programlisting>
1145 /usr/bin/g++-snapshot
1146 time g++
1147 <!-- Is this meant to be a single command? If not, insert "or" -->
1148 </programlisting>
1149 as the command. Second, while some tools have a logical
1150 "installation root", it's better if the user doesn't have to remember whether
1151 a specific tool requires a full command or a path.
1152 <!-- But many tools are really collections: e.g. a
1153 compiler, a linker, and others. The idea that the
1154 "command to invoke" has any significance may be
1155 completely bogus. Plus if you want to allow "time
1156 /usr/bin/g++" the toolset may need to somehow parse
1157 the command and find the path when it needs to invoke
1158 some related executable. And in that case, will the
1159 command be ignored? This scheme doesn't scale and
1160 should be fixed. -->
1161 </para></listitem>
1162
1163 <listitem><para>Check for multiple initialization. A user can try to
1164 initialize the module several times. You need to check for this
1165 and decide what to do. Typically, unless you support several
1166 versions of a tool, duplicate initialization is a user error.
1167 <!-- Why should that be typical? -->
1168 If the
1169 tool's version can be specified during initialization, make sure the
1170 version is either always specified, or never specified (in which
1171 case the tool is initialied only once). For example, if you allow:
1172 <programlisting>
1173 using yfc ;
1174 using yfc : 3.3 ;
1175 using yfc : 3.4 ;
1176 </programlisting>
1177 Then it's not clear if the first initialization corresponds to
1178 version 3.3 of the tool, version 3.4 of the tool, or some other
1179 version. This can lead to building twice with the same version.
1180 <!-- That would not be so terrible, and is much less harmful
1181 than this restriction, IMO. It makes site-config
1182 harder to maintain than necessary. -->
1183 </para></listitem>
1184
1185 <listitem><para>If possible, <code>init</code> must be callable
1186 with no parameters. In which case, it should try to autodetect all
1187 the necessary information, for example, by looking for a tool in
1188 <envar>PATH</envar> or in common installation locations. Often this
1189 is possible and allows the user to simply write:
1190 <programlisting>
1191 using yfc ;
1192 </programlisting>
1193 </para></listitem>
1194
1195 <listitem><para>Consider using facilities in the
1196 <code>tools/common</code> module. You can take a look at how
1197 <code>tools/gcc.jam</code> uses that module in the <code>init</code> rule.
1198 </para></listitem>
1199
1200 </itemizedlist>
1201 </para>
1202
1203
1204
1205
1206 </section>
1207
1208 </chapter>
1209
1210 <!--
1211 Local Variables:
1212 sgml-indent-data: t
1213 sgml-parent-document: ("userman.xml" "chapter")
1214 sgml-set-face: t
1215 End:
1216 -->