.. Copyright David Abrahams 2006. Distributed under the Boost
.. Software License, Version 1.0. (See accompanying
.. file LICENSE_1_0.txt or copy at
.. http://www.boost.org/LICENSE_1_0.txt)
==============================================
|(logo)|__ Boost.Python Build and Test HOWTO
==============================================
.. |(logo)| image:: ../../../boost.png
:alt: Boost C++ Libraries:
:class: boost-logo
__ ../index.htm
.. section-numbering::
:depth: 2
.. contents:: Contents
:depth: 2
:class: sidebar small
.. |newer| replace:: *newer*
Requirements
============
Boost.Python requires `Python 2.2`_ [#2.2]_ *or* |newer|__.
.. _Python 2.2: http://www.python.org/2.2
__ http://www.python.org
Background
==========
There are two basic models for combining C++ and Python:
- extending_, in which the end-user launches the Python interpreter
executable and imports Python “extension modules” written in C++.
Think of taking a library written in C++ and giving it a Python
interface so Python programmers can use it. From Python, these
modules look just like regular Python modules.
- embedding_, in which the end-user launches a program written
in C++ that in turn invokes the Python interpreter as a library
subroutine. Think of adding scriptability to an existing
application.
.. _extending: http://www.python.org/doc/current/ext/intro.html
.. _embedding: http://www.python.org/doc/current/ext/embedding.html
The key distinction between extending and embedding is the location
of the C++ ``main()`` function: in the Python interpreter executable,
or in some other program, respectively. Note that even when
embedding Python in another program, `extension modules are often
the best way to make C/C++ functionality accessible to Python
code`__, so the use of extension modules is really at the heart of
both models.
__ http://www.python.org/doc/current/ext/extending-with-embedding.html
Except in rare cases, extension modules are built as
dynamically-loaded libraries with a single entry point, which means
you can change them without rebuilding either the other extension
modules or the executable containing ``main()``.
.. _quickstart:
No-Install Quickstart
=====================
There is no need to “install Boost” in order to get started using
Boost.Python. These instructions use Boost.Build_ projects,
which will build those binaries as soon as they're needed. Your
first tests may take a little longer while you wait for
Boost.Python to build, but doing things this way will save you from
worrying about build intricacies like which library binaries to use
for a specific compiler configuration and figuring out the right
compiler options to use yourself.
.. .. raw:: html
.. Note:: Of course it's possible to use other build systems to
build Boost.Python and its extensions, but they are not
officially supported by Boost. Moreover **99% of all “I can't
build Boost.Python” problems come from trying to use another
build system** without first following these instructions.
If you want to use another system anyway, we suggest that you
follow these instructions, and then invoke ``bjam`` with the
.. parsed-literal::
``-a -o``\ *filename*
options to dump the build commands it executes to a file, so
you can see what your alternate build system needs to do.
.. .. raw:: html
.. _Boost.Build: ../../../tools/build/index.html
Basic Procedure
---------------
1. Get Boost; see sections 1 and 2 [`Unix/Linux`__, `Windows`__\ ] of the
Boost `Getting Started Guide`_.
__ ../../../more/getting_started/unix-variants.html#get-boost
__ ../../../more/getting_started/windows.html#get-boost
2. Get the ``bjam`` build driver. See section 5 [`Unix/Linux`__,
`Windows`__\ ] of the Boost `Getting Started Guide`_.
__ ../../../more/getting_started/unix-variants.html#prepare-to-use-a-boost-library-binary
__ ../../../more/getting_started/windows.html#prepare-to-use-a-boost-library-binary
3. cd into the ``libs/python/example/quickstart/`` directory of your
Boost installation, which contains a small example project.
4. Invoke ``bjam``. Replace the “\ ``stage``\ “ argument from the
example invocation from section 5 of the `Getting Started
Guide`_ with “\ ``test``\ ,“ to build all the test targets. Also add
the argument “\ ``--verbose-test``\ ” to see the output generated by
the tests when they are run.
On Windows, your ``bjam`` invocation might look something like:
.. parsed-literal::
C:\\boost_1_34_0\\…\\quickstart> **bjam toolset=msvc --verbose-test test**
and on Unix variants, perhaps,
.. parsed-literal::
~/boost_1_34_0/…/quickstart$ **bjam toolset=gcc --verbose-test test**
.. Admonition:: Note to Windows Users
For the sake of concision, the rest of this guide will use
unix-style forward slashes in pathnames instead of the
backslashes with which you may be more familiar. The forward
slashes should work everywhere except in `Command Prompt`_
windows, where you should use backslashes.
.. _Command Prompt: ../../../more/getting_started/windows.html#command-prompt
If you followed this procedure successfully, you will have built an
extension module called ``extending`` and tested it by running a
Python script called ``test_extending.py``. You will also have
built and run a simple application called ``embedding`` that embeds
python.
.. _Getting Started Guide: ../../../more/getting_started/index.html
In Case of Trouble
------------------
If you're seeing lots of compiler and/or linker error messages,
it's probably because Boost.Build is having trouble finding your
Python installation. You might want to pass the
``--debug-configuration`` option to ``bjam`` the first few times
you invoke it, to make sure that Boost.Build is correctly locating
all the parts of your Python installation. If it isn't, consider
`Configuring Boost.Build`_ as detailed below.
If you're still having trouble, Someone on one of the following
mailing lists may be able to help:
* The `Boost.Build mailing list`__ for issues related to Boost.Build
* The Python `C++ Sig`__ for issues specifically related to Boost.Python
__ ../../../more/mailing_lists.htm#jamboost
__ ../../../more/mailing_lists.htm#cplussig
In Case Everything Seemed to Work
---------------------------------
Rejoice! If you're new to Boost.Python, at this point it might be
a good idea to ignore build issues for a while and concentrate on
learning the library by going through the tutorial_ and perhaps
some of the `reference documentation`_, trying out what you've
learned about the API by modifying the quickstart project.
.. _reference documentation: v2/reference.html
.. _tutorial: tutorial/index.html
Modifying the Example Project
-----------------------------
If you're content to keep your extension module forever in one
source file called |extending.cpp|_, inside your Boost
distribution, and import it forever as ``extending``, then you can
stop here. However, it's likely that you will want to make a few
changes. There are a few things you can do without having to learn
Boost.Build_ in depth.
The project you just built is specified in two files in the current
directory: |boost-build.jam|_, which tells ``bjam`` where it can
find the interpreted code of the Boost build system, and
|Jamroot|_, which describes the targets you just built. These
files are heavily commented, so they should be easy to modify.
Take care, however, to preserve whitespace. Punctuation such as
``;`` will not be recognized as intended by ``bjam`` if it is not
surrounded by whitespace.
.. |boost-build.jam| replace:: ``boost-build.jam``
.. _boost-build.jam: ../example/quickstart/boost-build.jam
.. |Jamroot| replace:: ``Jamroot``
.. _Jamroot: ../example/quickstart/Jamroot
.. |extending.cpp| replace:: ``extending.cpp``
.. _extending.cpp: ../example/quickstart/extending.cpp
Relocate the Project
....................
You'll probably want to copy this project elsewhere so you can
change it without modifying your Boost distribution. To do that,
simply
a. copy the entire ``libs/python/example/quickstart/`` directory
into a new directory.
b. In the new copies of |boost-build.jam|_ and |Jamroot|_, locate
the relative path near the top of the file that is clearly
marked by a comment, and edit that path so that it refers to the
same directory your Boost distribution as it referred to when
the file was in its original location in the
``libs/python/example/quickstart/`` directory.
For example, if you moved the project from
``/home/dave/boost_1_34_0/libs/python/example/quickstart`` to
``/home/dave/my-project``, you could change the first path in
|boost-build.jam|_ from
.. parsed-literal::
**../../../..**\ /tools/build/v2
to
.. parsed-literal::
**/home/dave/boost_1_34_0**\ /tools/build/v2
and change the first path in |Jamroot|_ from
.. parsed-literal::
**../../../..**
to
.. parsed-literal::
**/home/dave/boost_1_34_0**
Add New or Change Names of Existing Source Files
................................................
The names of additional source files involved in building your
extension module or embedding application can be listed in
|Jamroot|_ right alongside ``extending.cpp`` or ``embedding.cpp``
respectively. Just be sure to leave whitespace around each
filename::
… file1.cpp file2.cpp file3.cpp …
Naturally, if you want to change the name of a source file you can
tell Boost.Build about it by editing the name in |Jamroot|_.
Change the Name of your Extension Module
........................................
The name of the extension module is determined by two things:
1. the name in |Jamroot|_ immediately following ``python-extension``, and
2. the name passed to ``BOOST_PYTHON_MODULE`` in |extending.cpp|_.
To change the name of the extension module from ``extending`` to
``hello``, you'd edit |Jamroot|_, changing
.. parsed-literal::
python-extension **extending** : extending.cpp ;
to
.. parsed-literal::
python-extension **hello** : extending.cpp ;
and you'd edit extending.cpp, changing
.. parsed-literal::
BOOST_PYTHON_MODULE(\ **extending**\ )
to
.. parsed-literal::
BOOST_PYTHON_MODULE(\ **hello**\ )
Installing Boost.Python on your System
======================================
Since Boost.Python is a separately-compiled (as opposed to
`header-only`_) library, its user relies on the services of a
Boost.Python library binary.
.. _header-only: ../../../more/getting_started/windows.html#header-only-libraries
If you need a regular installation of the Boost.Python library
binaries on your system, the Boost `Getting Started Guide`_ will
walk you through the steps of creating one. If building binaries
from source, you might want to supply the ``--with-python``
argument to ``bjam`` (or the ``--with-libraries=python`` argument
to ``configure``), so only the Boost.Python binary will be built,
rather than all the Boost binaries.
Configuring Boost.Build
=======================
As described in the `Boost.Build reference manual`__, a file called
``user-config.jam`` in your home directory [#home-dir]_ is used to
specify the tools and libraries available to the build system. You
may need to create or edit ``user-config.jam`` to tell Boost.Build
how to invoke Python, ``#include`` its headers, and link with its
libraries.
__ http://www.boost.orgdoc/html/bbv2/advanced.html#bbv2.advanced.configuration
.. Admonition:: Users of Unix-Variant OSes
If you are using a unix-variant OS and you ran Boost's
``configure`` script, it may have generated a
``user-config.jam`` for you. [#overwrite]_ If your ``configure``\
/\ ``make`` sequence was successful and Boost.Python binaries
were built, your ``user-config.jam`` file is probably already
correct.
If you have one fairly “standard” python installation for your
platform, you might not need to do anything special to describe it. If
you haven't configured python in ``user-config.jam`` (and you don't
specify ``--without-python`` on the Boost.Build command line),
Boost.Build will automatically execute the equivalent of ::
import toolset : using ;
using python ;
which automatically looks for Python in the most likely places.
However, that only happens when using the Boost.Python project file
(e.g. when referred to by another project as in the quickstart_
method). If instead you are linking against separately-compiled
Boost.Python binaries, you should set up a ``user-config.jam`` file
with at least the minimal incantation above.
Python Configuration Parameters
-------------------------------
If you have several versions of Python installed, or Python is
installed in an unusual way, you may want to supply any or all of
the following optional parameters to ``using python``.
version
the version of Python to use. Should be in Major.Minor
format, for example, ``2.3``. Do not include the subminor
version (i.e. *not* ``2.5.1``). If you have multiple Python
versions installed, the version will usually be the only
configuration argument required.
cmd-or-prefix
preferably, a command that invokes a Python interpreter.
Alternatively, the installation prefix for Python libraries and
header files. Only use the alternative formulation if there is
no appropriate Python executable available.
includes
the ``#include`` paths for Python headers. Normally the correct
path(s) will be automatically deduced from ``version`` and/or
``cmd-or-prefix``.
libraries
the path to Python library binaries. On MacOS/Darwin,
you can also pass the path of the Python framework. Normally the
correct path(s) will be automatically deduced from ``version``
and/or ``cmd-or-prefix``.
condition
if specified, should be a set of Boost.Build
properties that are matched against the build configuration when
Boost.Build selects a Python configuration to use. See examples
below for details.
extension-suffix
A string to append to the name of extension
modules before the true filename extension. You almost certainly
don't need to use this. Usually this suffix is only used when
targeting a Windows debug build of Python, and will be set
automatically for you based on the value of the
|python-debugging|_ feature. However, at least one Linux
distribution (Ubuntu Feisty Fawn) has a specially configured
`python-dbg`__ package that claims to use such a suffix.
.. |python-debugging| replace:: ````
__ https://wiki.ubuntu.com/PyDbgBuilds
Examples
--------
Note that in the examples below, case and *especially whitespace* are
significant.
- If you have both python 2.5 and python 2.4 installed,
``user-config.jam`` might contain::
using python : 2.5 ; # Make both versions of Python available
using python : 2.4 ; # To build with python 2.4, add python=2.4
# to your command line.
The first version configured (2.5) becomes the default. To build
against python 2.4, add ``python=2.4`` to the ``bjam`` command line.
- If you have python installed in an unusual location, you might
supply the path to the interpreter in the ``cmd-or-prefix``
parameter::
using python : : /usr/local/python-2.6-beta/bin/python ;
- If you have a separate build of Python for use with a particular
toolset, you might supply that toolset in the ``condition``
parameter::
using python ; # use for most toolsets
# Use with Intel C++ toolset
using python
: # version
: c:\\Devel\\Python-2.5-IntelBuild\\PCBuild\\python # cmd-or-prefix
: # includes
: # libraries
: intel # condition
;
- If you have downloaded the Python sources and built both the
normal and the “\ `python debugging`_\ ” builds from source on
Windows, you might see::
using python : 2.5 : C:\\src\\Python-2.5\\PCBuild\\python ;
using python : 2.5 : C:\\src\\Python-2.5\\PCBuild\\python_d
: # includes
: # libs
: on ;
- You can set up your user-config.jam so a bjam built under Windows
can build/test both Windows and Cygwin_ python extensions. Just pass
``cygwin`` in the ``condition`` parameter
for the cygwin python installation::
# windows installation
using python ;
# cygwin installation
using python : : c:\\cygwin\\bin\\python2.5 : : : cygwin ;
when you put target-os=cygwin in your build request, it should build
with the cygwin version of python: [#flavor]_
bjam target-os=cygwin toolset=gcc
This is supposed to work the other way, too (targeting windows
python with a Cygwin_ bjam) but it seems as though the support in
Boost.Build's toolsets for building that way is broken at the
time of this writing.
- Note that because of `the way Boost.Build currently selects target
alternatives`__, you might have be very explicit in your build
requests. For example, given::
using python : 2.5 ; # a regular windows build
using python : 2.4 : : : : cygwin ;
building with ::
bjam target-os=cygwin
will yield an error. Instead, you'll need to write::
bjam target-os=cygwin/python=2.4
.. _Cygwin: http://cygwin.com
__ http://zigzag.cs.msu.su/boost.build/wiki/AlternativeSelection
Choosing a Boost.Python Library Binary
======================================
If—instead of letting Boost.Build construct and link with the right
libraries automatically—you choose to use a pre-built Boost.Python
library, you'll need to think about which one to link with. The
Boost.Python binary comes in both static and dynamic flavors. Take
care to choose the right flavor for your application. [#naming]_
The Dynamic Binary
------------------
The dynamic library is the safest and most-versatile choice:
- A single copy of the library code is used by all extension
modules built with a given toolset. [#toolset-specific]_
- The library contains a type conversion registry. Because one
registry is shared among all extension modules, instances of a
class exposed to Python in one dynamically-loaded extension
module can be passed to functions exposed in another such module.
The Static Binary
-----------------
It might be appropriate to use the static Boost.Python library in
any of the following cases:
- You are extending_ python and the types exposed in your
dynamically-loaded extension module don't need to be used by any
other Boost.Python extension modules, and you don't care if the
core library code is duplicated among them.
- You are embedding_ python in your application and either:
- You are targeting a Unix variant OS other than MacOS or AIX,
where the dynamically-loaded extension modules can “see” the
Boost.Python library symbols that are part of the executable.
- Or, you have statically linked some Boost.Python extension
modules into your application and you don't care if any
dynamically-loaded Boost.Python extension modules are able to
use the types exposed by your statically-linked extension
modules (and vice-versa).
``#include`` Issues
===================
1. If you should ever have occasion to ``#include "python.h"``
directly in a translation unit of a program using Boost.Python,
use ``#include "boost/python/detail/wrap_python.hpp"`` instead.
It handles several issues necessary for use with Boost.Python,
one of which is mentioned in the next section.
2. Be sure not to ``#include`` any system headers before
``wrap_python.hpp``. This restriction is actually imposed by
Python, or more properly, by Python's interaction with your
operating system. See
http://docs.python.org/ext/simpleExample.html for details.
.. _python-debugging:
.. _python debugging:
Python Debugging Builds
=======================
Python can be built in a special “python debugging” configuration
that adds extra checks and instrumentation that can be very useful
for developers of extension modules. The data structures used by
the debugging configuration contain additional members, so **a
Python executable built with python debugging enabled cannot be
used with an extension module or library compiled without it, and
vice-versa.**
Since pre-built “python debugging” versions of the Python
executable and libraries are not supplied with most distributions
of Python, [#get-debug-build]_ and we didn't want to force our users
to build them, Boost.Build does not automatically enable python
debugging in its ``debug`` build variant (which is the default).
Instead there is a special build property called
``python-debugging`` that, when used as a build property, will
define the right preprocessor symbols and select the right
libraries to link with.
On unix-variant platforms, the debugging versions of Python's data
structures will only be used if the symbol ``Py_DEBUG`` is defined.
On many windows compilers, when extension modules are built with
the preprocessor symbol ``_DEBUG``, Python defaults to force
linking with a special debugging version of the Python DLL. Since
that symbol is very commonly used even when Python is not present,
Boost.Python temporarily undefines _DEBUG when Python.h
is #included from ``boost/python/detail/wrap_python.hpp`` - unless
``BOOST_DEBUG_PYTHON`` is defined. The upshot is that if you want
“python debugging”and you aren't using Boost.Build, you should make
sure ``BOOST_DEBUG_PYTHON`` is defined, or python debugging will be
suppressed.
Testing Boost.Python
====================
To run the full test suite for Boost.Python, invoke ``bjam`` in the
``libs/python/test`` subdirectory of your Boost distribution.
Notes for MinGW (and Cygwin with -mno-cygwin) GCC Users
=======================================================
If you are using a version of Python prior to 2.4.1 with a MinGW
prior to 3.0.0 (with binutils-2.13.90-20030111-1), you will need to
create a MinGW-compatible version of the Python library; the one
shipped with Python will only work with a Microsoft-compatible
linker. Follow the instructions in the “Non-Microsoft” section of
the “Building Extensions: Tips And Tricks” chapter in `Installing
Python Modules`__ to create ``libpythonXX.a``, where ``XX``
corresponds to the major and minor version numbers of your Python
installation.
__ http://www.python.org/doc/current/inst/index.html
-----------------------------
.. [#2.2] Note that although we tested earlier versions of
Boost.Python with Python 2.2, and we don't *think* we've done
anything to break compatibility, this release of Boost.Python
may not have been tested with versions of Python earlier than
2.4, so we're not 100% sure that python 2.2 and 2.3 are
supported.
.. [#naming] Information about how to identify the
static and dynamic builds of Boost.Python:
* `on Windows`__
* `on Unix variants`__
__ ../../../more/getting_started/windows.html#library-naming
__ ../../../more/getting_started/unix-variants.html#library-naming
.. [#toolset-specific] Because of the way most \*nix platforms
share symbols among dynamically-loaded objects, I'm not certain
that extension modules built with different compiler toolsets
will always use different copies of the Boost.Python library
when loaded into the same Python instance. Not using different
libraries could be a good thing if the compilers have compatible
ABIs, because extension modules built with the two libraries
would be interoperable. Otherwise, it could spell disaster,
since an extension module and the Boost.Python library would
have different ideas of such things as class layout. I would
appreciate someone doing the experiment to find out what
happens.
.. [#overwrite] ``configure`` overwrites the existing
``user-config.jam`` in your home directory
(if any) after making a backup of the old version.
.. [#flavor] Note that the ``cygwin`` feature is
different from the ``cygwin`` subfeature of the ``gcc``
toolset, and you might need handle both explicitly if you also
have a MinGW GCC installed.
.. [#home-dir] Windows users, your home directory can be
found by typing::
ECHO %HOMEDRIVE%%HOMEPATH%
into a `command prompt`_ window.
.. [#get-debug-build] On Unix and similar platforms, a debugging
python and associated libraries are built by adding
``--with-pydebug`` when configuring the Python build. On
Windows, the debugging version of Python is generated by
the "Win32 Debug" target of the Visual Studio project in the
PCBuild subdirectory of a full Python source code distribution.
You may also find