1 [chapter Building and Testing
3 [authors [Abrahams, David]]
4 [copyright 2002 - 2015 David Abrahams, Stefan Seefeld]
7 [/ Copyright David Abrahams 2006. Distributed under the Boost
8 / Software License, Version 1.0. (See accompanying
9 / file LICENSE_1_0.txt or copy at
10 / http://www.boost.org/LICENSE_1_0.txt)
13 [section Requirements]
15 Boost.Python requires [@http://www.python.org/2.2 Python 2.2]
16 [footnote Note that although we tested earlier versions of Boost.Python
17 with Python 2.2, and we don't *think* we've done anything to break
18 compatibility, this release of Boost.Python may not have been tested
19 with versions of Python earlier than 2.4, so we're not 100% sure that
20 python 2.2 and 2.3 are supported.] *or* [@http://www.python.org newer].
25 There are two basic models for combining C++ and Python:
27 * [@http://www.python.org/doc/current/ext/intro.html extending],
28 in which the end-user launches the Python interpreter
29 executable and imports Python “extension modules” written in C++.
30 Think of taking a library written in C++ and giving it a Python
31 interface so Python programmers can use it. From Python, these
32 modules look just like regular Python modules.
34 * [@http://www.python.org/doc/current/ext/embedding.html embedding],
35 in which the end-user launches a program written
36 in C++ that in turn invokes the Python interpreter as a library
37 subroutine. Think of adding scriptability to an existing
40 The key distinction between extending and embedding is the location
41 of the C++ `main()` function: in the Python interpreter executable,
42 or in some other program, respectively. Note that even when
43 embedding Python in another program, [@http://www.python.org/doc/current/ext/extending-with-embedding.html extension modules are often
44 the best way to make C/C++ functionality accessible to Python
45 code], so the use of extension modules is really at the heart of
48 Except in rare cases, extension modules are built as
49 dynamically-loaded libraries with a single entry point, which means
50 you can change them without rebuilding either the other extension
51 modules or the executable containing `main()`.
54 [section No-Install Quickstart]
56 There is no need to “install Boost” in order to get started using
57 Boost.Python. These instructions use _bb_ projects,
58 which will build those binaries as soon as they're needed. Your
59 first tests may take a little longer while you wait for
60 Boost.Python to build, but doing things this way will save you from
61 worrying about build intricacies like which library binaries to use
62 for a specific compiler configuration and figuring out the right
63 compiler options to use yourself.
65 [note Of course it's possible to use other build systems to
66 build Boost.Python and its extensions, but they are not
67 officially supported by Boost. Moreover *99% of all “I can't
68 build Boost.Python” problems come from trying to use another
69 build system* without first following these instructions.
71 If you want to use another system anyway, we suggest that you
72 follow these instructions, and then invoke `bjam` with the
76 options to dump the build commands it executes to a file, so
77 you can see what your alternate build system needs to do.]
79 [section Basic Procedure]
81 1. Get Boost; see sections 1 and 2 of the _gsg_.
83 2. Get the `bjam` build driver. See section 5 of the _gsg_.
85 3. cd into the `example/quickstart/` directory of your
86 Boost.Python installation, which contains a small example project.
88 4. Invoke `bjam`. Replace the “\ `stage`\ “ argument from the
89 example invocation from section 5 of the _gsg_ with “\ `test`\ ,“ to
90 build all the test targets. Also add the argument “\ `--verbose-test`\ ”
91 to see the output generated by the tests when they are run.
92 On Windows, your `bjam` invocation might look something like:
94 C:\\...\\quickstart> bjam toolset=msvc --verbose-test test
96 and on Unix variants, perhaps,
98 .../quickstart$ bjam toolset=gcc --verbose-test test
101 [note For the sake of concision, the rest of this guide will use
102 unix-style forward slashes in pathnames instead of the
103 backslashes with which Windows users may be more familiar. The forward
104 slashes should work everywhere except in
105 [@http://www.boost.org/more/getting_started/windows.html#command-prompt
106 Command Prompt] windows, where you should use backslashes.]
108 If you followed this procedure successfully, you will have built an
109 extension module called `extending` and tested it by running a
110 Python script called `test_extending.py`. You will also have
111 built and run a simple application called `embedding` that embeds
115 [section In Case of Trouble]
117 If you're seeing lots of compiler and/or linker error messages,
118 it's probably because Boost.Build is having trouble finding your
119 Python installation. You might want to pass the
120 `--debug-configuration` option to `bjam` the first few times
121 you invoke it, to make sure that Boost.Build is correctly locating
122 all the parts of your Python installation. If it isn't, consider
123 [link building.configuring_boost_build Configuring Boost.Build]
126 If you're still having trouble, Someone on one of the following
127 mailing lists may be able to help:
129 * The _bb_list_ for issues related to Boost.Build
130 * The _bp_list_ for issues specifically related to Boost.Python
133 [section In Case Everything Seemed to Work]
135 Rejoice! If you're new to Boost.Python, at this point it might be
136 a good idea to ignore build issues for a while and concentrate on
137 learning the library by going through the _tutorial_ and perhaps
138 some of the _reference_, trying out what you've
139 learned about the API by modifying the quickstart project.
142 [section Modifying the Example Project]
144 If you're content to keep your extension module forever in one
145 source file called `extending.cpp`, inside your Boost.Python
146 distribution, and import it forever as `extending`, then you can
147 stop here. However, it's likely that you will want to make a few
148 changes. There are a few things you can do without having to learn
151 The project you just built is specified in two files in the current
152 directory: `boost-build.jam`, which tells `bjam` where it can
153 find the interpreted code of the Boost build system, and
154 `Jamroot`, which describes the targets you just built. These
155 files are heavily commented, so they should be easy to modify.
156 Take care, however, to preserve whitespace. Punctuation such as
157 `;` will not be recognized as intended by `bjam` if it is not
158 surrounded by whitespace.
160 [section Relocate the Project]
162 You'll probably want to copy this project elsewhere so you can
163 change it without modifying your Boost distribution. To do that,
166 a. copy the entire `example/quickstart/` directory
167 into a new directory.
169 b. In the new copies of `boost-build.jam` and `Jamroot`, locate
170 the relative path near the top of the file that is clearly
171 marked by a comment, and edit that path so that it refers to the
172 same directory your Boost distribution as it referred to when
173 the file was in its original location in the
174 `example/quickstart/` directory.
176 For example, if you moved the project from
177 `/home/dave/boost_1_34_0/libs/python/example/quickstart` to
178 `/home/dave/my-project`, you could change the first path in
179 `boost-build.jam` from
181 ../../../../tools/build/src
185 /home/dave/boost_1_34_0/tools/build/src
187 and change the first path in `Jamroot` from
193 /home/dave/boost_1_34_0
197 [section Add New or Change Names of Existing Source Files]
199 The names of additional source files involved in building your
200 extension module or embedding application can be listed in
201 `Jamroot` right alongside `extending.cpp` or `embedding.cpp`
202 respectively. Just be sure to leave whitespace around each
205 … file1.cpp file2.cpp file3.cpp …
207 Naturally, if you want to change the name of a source file you can
208 tell Boost.Build about it by editing the name in `Jamroot`.
211 [section Change the Name of your Extension Module]
213 The name of the extension module is determined by two things:
215 # the name in `Jamroot` immediately following `python-extension`, and
216 # the name passed to `BOOST_PYTHON_MODULE` in `extending.cpp`.
218 To change the name of the extension module from `extending` to
219 `hello`, you'd edit `Jamroot`, changing
221 python-extension extending : extending.cpp ;
225 python-extension hello : extending.cpp ;
227 and you'd edit extending.cpp, changing
230 BOOST_PYTHON_MODULE(extending)
234 BOOST_PYTHON_MODULE(hello)
239 [section Installing Boost.Python on your System]
241 Since Boost.Python is a separately-compiled (as opposed to
242 `header-only`) library, its user relies on the services of a
243 Boost.Python library binary.
245 If you need a regular installation of the Boost.Python library
246 binaries on your system, the _gsg_ will
247 walk you through the steps of creating one. If building binaries
248 from source, you might want to supply the `--with-python`
249 argument to `bjam` (or the `--with-libraries=python` argument
250 to `configure`), so only the Boost.Python binary will be built,
251 rather than all the Boost binaries.
254 [section Configuring Boost.Build]
256 As described in the [@http://www.boost.org/build/doc/html/bbv2/overview/configuration.html Boost.Build Reference Manual], a file called
257 `user-config.jam` in your home directory is used to
258 specify the tools and libraries available to the build system. You
259 may need to create or edit `user-config.jam` to tell Boost.Build
260 how to invoke Python, `#include` its headers, and link with its
263 [note If you are using a unix-variant OS and you ran Boost's
264 `configure` script, it may have generated a
265 `user-config.jam` for you. [footnote `configure` overwrites the existing
266 `user-config.jam` in your home directory (if any) after making a backup of
267 the old version.] If your `configure`\ /\ `make` sequence was successful and
268 Boost.Python binaries were built, your `user-config.jam` file is probably already
271 If you have one fairly “standard” python installation for your
272 platform, you might not need to do anything special to describe it. If
273 you haven't configured python in `user-config.jam` (and you don't
274 specify `--without-python` on the Boost.Build command line),
275 Boost.Build will automatically execute the equivalent of
278 import toolset : using ;
281 which automatically looks for Python in the most likely places.
282 However, that only happens when using the Boost.Python project file
283 (e.g. when referred to by another project as in the quickstart
284 method). If instead you are linking against separately-compiled
285 Boost.Python binaries, you should set up a `user-config.jam` file
286 with at least the minimal incantation above.
288 [section Python Configuration Parameters]
290 If you have several versions of Python installed, or Python is
291 installed in an unusual way, you may want to supply any or all of
292 the following optional parameters to `using python`.
297 [the version of Python to use. Should be in Major.Minor
298 format, for example, `2.3`. Do not include the subminor
299 version (i.e. *not* `2.5.1`). If you have multiple Python
300 versions installed, the version will usually be the only
301 configuration argument required.]]
305 [preferably, a command that invokes a Python interpreter.
306 Alternatively, the installation prefix for Python libraries and
307 header files. Only use the alternative formulation if there is
308 no appropriate Python executable available.]]
312 [the `#include` paths for Python headers. Normally the correct
313 path(s) will be automatically deduced from `version` and/or
318 [the path to Python library binaries. On MacOS/Darwin,
319 you can also pass the path of the Python framework. Normally the
320 correct path(s) will be automatically deduced from `version`
321 and/or `cmd-or-prefix`.]]
325 [if specified, should be a set of Boost.Build
326 properties that are matched against the build configuration when
327 Boost.Build selects a Python configuration to use. See examples
330 [[*extension-suffix*]
332 [A string to append to the name of extension
333 modules before the true filename extension. You almost certainly
334 don't need to use this. Usually this suffix is only used when
335 targeting a Windows debug build of Python, and will be set
336 automatically for you based on the value of the
337 [link building.python_debugging_builds <python-debugging>] feature.
338 However, at least one Linux distribution (Ubuntu Feisty Fawn) has
339 a specially configured [@https://wiki.ubuntu.com/PyDbgBuilds <python-dbg>]
340 package that claims to use such a suffix.]]
346 Note that in the examples below, case and *especially whitespace* are
349 * If you have both python 2.5 and python 2.4 installed,
350 `user-config.jam` might contain
353 using python : 2.5 ; # Make both versions of Python available
354 using python : 2.4 ; # To build with python 2.4, add python=2.4
355 # to your command line.
357 The first version configured (2.5) becomes the default. To build
358 against python 2.4, add `python=2.4` to the `bjam` command line.
360 * If you have python installed in an unusual location, you might
361 supply the path to the interpreter in the `cmd-or-prefix`
365 using python : : /usr/local/python-2.6-beta/bin/python ;
368 * If you have a separate build of Python for use with a particular
369 toolset, you might supply that toolset in the `condition`
373 using python ; # use for most toolsets
375 # Use with Intel C++ toolset
378 : c:\\Devel\\Python-2.5-IntelBuild\\PCBuild\\python # cmd-or-prefix
381 : <toolset>intel # condition
385 * If you have downloaded the Python sources and built both the
386 normal and the [link building.python_debugging_builds "python debugging"]
387 builds from source on Windows, you might see:
390 using python : 2.5 : C:\\src\\Python-2.5\\PCBuild\\python ;
391 using python : 2.5 : C:\\src\\Python-2.5\\PCBuild\\python_d
394 : <python-debugging>on ;
396 * You can set up your user-config.jam so a bjam built under Windows
397 can build/test both Windows and Cygwin_ python extensions. Just pass
398 `<target-os>cygwin` in the `condition` parameter
399 for the cygwin python installation:
402 # windows installation
405 # cygwin installation
406 using python : : c:\\cygwin\\bin\\python2.5 : : : <target-os>cygwin ;
408 when you put target-os=cygwin in your build request, it should build
409 with the cygwin version of python: [#flavor]_
412 bjam target-os=cygwin toolset=gcc
414 This is supposed to work the other way, too (targeting windows
415 python with a [@http://cygwin.com Cygwin] bjam) but it seems as though the support in
416 Boost.Build's toolsets for building that way is broken at the
417 time of this writing.
419 * Note that because of [@http://zigzag.cs.msu.su/boost.build/wiki/AlternativeSelection
420 the way Boost.Build currently selects target alternatives], you might have be very
421 explicit in your build requests. For example, given:
424 using python : 2.5 ; # a regular windows build
425 using python : 2.4 : : : : <target-os>cygwin ;
429 bjam target-os=cygwin
432 will yield an error. Instead, you'll need to write
435 bjam target-os=cygwin/python=2.4
440 [section Choosing a Boost.Python Library Binary]
442 If—instead of letting Boost.Build construct and link with the right
443 libraries automatically—you choose to use a pre-built Boost.Python
444 library, you'll need to think about which one to link with. The
445 Boost.Python binary comes in both static and dynamic flavors. Take
446 care to choose the right flavor for your application. [footnote
447 Information about how to identify the static and dynamic builds of Boost.Python on
448 [@http://boost.org/more/getting_started/windows.html#library-naming Windows] /
449 [@http://boost.org/more/getting_started/unix-variants.html#library-naming Unix variants]]
451 [section The Dynamic Binary]
453 The dynamic library is the safest and most-versatile choice:
455 * A single copy of the library code is used by all extension
456 modules built with a given toolset. [footnote Because of the way most \*nix platforms
457 share symbols among dynamically-loaded objects, I'm not certain
458 that extension modules built with different compiler toolsets
459 will always use different copies of the Boost.Python library
460 when loaded into the same Python instance. Not using different
461 libraries could be a good thing if the compilers have compatible
462 ABIs, because extension modules built with the two libraries
463 would be interoperable. Otherwise, it could spell disaster,
464 since an extension module and the Boost.Python library would
465 have different ideas of such things as class layout. I would
466 appreciate someone doing the experiment to find out what
469 * The library contains a type conversion registry. Because one
470 registry is shared among all extension modules, instances of a
471 class exposed to Python in one dynamically-loaded extension
472 module can be passed to functions exposed in another such module.
475 [section The Static Binary]
477 It might be appropriate to use the static Boost.Python library in
478 any of the following cases:
480 * You are _extending_ python and the types exposed in your
481 dynamically-loaded extension module don't need to be used by any
482 other Boost.Python extension modules, and you don't care if the
483 core library code is duplicated among them.
485 * You are _embedding_ python in your application and either:
487 * You are targeting a Unix variant OS other than MacOS or AIX,
488 where the dynamically-loaded extension modules can “see” the
489 Boost.Python library symbols that are part of the executable.
491 * Or, you have statically linked some Boost.Python extension
492 modules into your application and you don't care if any
493 dynamically-loaded Boost.Python extension modules are able to
494 use the types exposed by your statically-linked extension
495 modules (and vice-versa).
499 [section `#include` Issues]
501 1. If you should ever have occasion to `#include "python.h"`
502 directly in a translation unit of a program using Boost.Python,
503 use `#include "boost/python/detail/wrap_python.hpp"` instead.
504 It handles several issues necessary for use with Boost.Python,
505 one of which is mentioned in the next section.
507 2. Be sure not to `#include` any system headers before
508 `wrap_python.hpp`. This restriction is actually imposed by
509 Python, or more properly, by Python's interaction with your
510 operating system. See
511 [@http://docs.python.org/ext/simpleExample.html] for details.
514 [section Python Debugging Builds]
516 Python can be built in a special “python debugging” configuration
517 that adds extra checks and instrumentation that can be very useful
518 for developers of extension modules. The data structures used by
519 the debugging configuration contain additional members, so *a
520 Python executable built with python debugging enabled cannot be
521 used with an extension module or library compiled without it, and
524 Since pre-built “python debugging” versions of the Python
525 executable and libraries are not supplied with most distributions
526 of Python, [footnote On Unix and similar platforms, a debugging python and associated libraries are built by adding --with-pydebug when configuring the Python build. On Windows, the debugging version of Python is generated by the "Win32 Debug" target of the Visual Studio project in the PCBuild subdirectory of a full Python source code distribution.] and we didn't want to force our users
527 to build them, Boost.Build does not automatically enable python
528 debugging in its `debug` build variant (which is the default).
529 Instead there is a special build property called
530 `python-debugging` that, when used as a build property, will
531 define the right preprocessor symbols and select the right
532 libraries to link with.
534 On unix-variant platforms, the debugging versions of Python's data
535 structures will only be used if the symbol `Py_DEBUG` is defined.
536 On many windows compilers, when extension modules are built with
537 the preprocessor symbol `_DEBUG`, Python defaults to force
538 linking with a special debugging version of the Python DLL. Since
539 that symbol is very commonly used even when Python is not present,
540 Boost.Python temporarily undefines `_DEBUG` when `Python.h`
541 is #included from `boost/python/detail/wrap_python.hpp` - unless
542 `BOOST_DEBUG_PYTHON` is defined. The upshot is that if you want
543 “python debugging”and you aren't using Boost.Build, you should make
544 sure `BOOST_DEBUG_PYTHON` is defined, or python debugging will be
548 [section Testing Boost.Python]
550 To run the full test suite for Boost.Python, invoke `bjam` in the
551 `test` subdirectory of your Boost.Python distribution.
554 [section Notes for MinGW (and Cygwin with -mno-cygwin) GCC Users]
556 If you are using a version of Python prior to 2.4.1 with a MinGW
557 prior to 3.0.0 (with binutils-2.13.90-20030111-1), you will need to
558 create a MinGW-compatible version of the Python library; the one
559 shipped with Python will only work with a Microsoft-compatible
560 linker. Follow the instructions in the “Non-Microsoft” section of
561 the “Building Extensions: Tips And Tricks” chapter in
562 [@https://docs.python.org/2/install/index.html Installing Python Modules]
563 to create `libpythonXX.a`, where `XX` corresponds to the major and minor
564 version numbers of your Python installation.