Appendix¶
MPI-enabled Python interpreter¶
Warning
These days it is no longer required to use the MPI-enabled Python interpreter in most cases, and, therefore, is not built by default anymore because it is too difficult to reliably build a Python interpreter across different distributions. If you know that you still really need it, see below on how to use the build_exe and install_exe commands.
Some MPI-1 implementations (notably, MPICH 1) do require the
actual command line arguments to be passed at the time
MPI_Init()
is called. In this case, you will need to use a
re-built, MPI-enabled, Python interpreter binary executable. A basic
implementation (targeting Python 2.X) of what is required is shown
below:
#include <Python.h>
#include <mpi.h>
int main(int argc, char *argv[])
{
int status, flag;
MPI_Init(&argc, &argv);
status = Py_Main(argc, argv);
MPI_Finalized(&flag);
if (!flag) MPI_Finalize();
return status;
}
The source code above is straightforward; compiling it should also be. However, the linking step is more tricky: special flags have to be passed to the linker depending on your platform. In order to alleviate you for such low-level details, MPI for Python provides some pure-distutils based support to build and install an MPI-enabled Python interpreter executable:
$ cd mpi4py-X.X.X
$ python setup.py build_exe [--mpi=<name>|--mpicc=/path/to/mpicc]
$ [sudo] python setup.py install_exe [--install-dir=$HOME/bin]
After the above steps you should have the MPI-enabled interpreter
installed as prefix/bin/pythonX.X-mpi
(or
$HOME/bin/pythonX.X-mpi
). Assuming that
prefix/bin
(or $HOME/bin
) is listed on your
PATH
, you should be able to enter your MPI-enabled Python
interactively, for example:
$ python2.7-mpi
Python 2.7.8 (default, Nov 10 2014, 08:19:18)
[GCC 4.9.2 20141101 (Red Hat 4.9.2-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/usr/bin/python2.7-mpi'
>>>
Mac OS X and Universal/SDK Python builds¶
Mac OS X users employing a Python distribution built with support for
Universal applications could have
trouble building MPI for Python, specially if they want to link
against MPI libraries built without such support. Another source of
trouble could be a Python build using a specific deployment target
and cross-development SDK configuration. Workarounds for such issues
are to temporarily set the environment variables
MACOSX_DEPLOYMENT_TARGET
, SDKROOT
and/or
ARCHFLAGS
to appropriate values in the shell before trying
to build/install MPI for Python.
An appropriate value for MACOSX_DEPLOYMENT_TARGET
should be
any greater or equal than the one used to build Python, and less or
equal than your system version. The safest choice for end-users would
be to use the system version (e.g, if you are on Leopard, you should
try MACOSX_DEPLOYMENT_TARGET=10.5
).
An appropriate value for SDKROOT
is the full path name of
any of the SDK’s you have at /Developer/SDKs
directory (e.g.,
SDKROOT=/Developer/SDKs/MacOSX10.5.sdk
). The safest choice for
end-users would be the one matching the system version; or
alternatively the root directory (i.e., SDKROOT=/
).
Appropriate values for ARCHFLAGS
have the form -arch
<value>
, where <value>
should be chosen from the following
table:
@ | Intel | PowerPC |
---|---|---|
32-bit | i386 |
ppc |
64-bit | x86_64 |
ppc64 |
For example, assuming your Mac is running Snow Leopard on a 64-bit Intel processor and you want to override the hard-wired cross-development SDK in Python configuration, you can build and install MPI for Python using any of the alternatives below. Note that environment variables may need to be passed/set both at the build and install steps (because sudo may not pass environment variables to subprocesses for security reasons)
Alternative 1:
$ env MACOSX_DEPLOYMENT_TARGET=10.6 \ SDKROOT=/ \ ARCHFLAGS='-arch x86_64' \ python setup.py build [options] $ sudo env MACOSX_DEPLOYMENT_TARGET=10.6 \ SDKROOT=/ \ ARCHFLAGS='-arch x86_64' \ python setup.py install [options]
Alternative 2:
$ export MACOSX_DEPLOYMENT_TARGET=10.6 $ export SDKROOT=/ $ export ARCHFLAGS='-arch x86_64' $ python setup.py build [options] $ sudo -s # enter interactive shell as root $ export MACOSX_DEPLOYMENT_TARGET=10.6 $ export SDKROOT=/ $ export ARCHFLAGS='-arch x86_64' $ python setup.py install [options] $ exit
Building MPI from sources¶
In the list below you have some executive instructions for building some of the open-source MPI implementations out there with support for shared/dynamic libraries on POSIX environments.
MPICH
$ tar -zxf mpich-X.X.X.tar.gz $ cd mpich-X.X.X $ ./configure --enable-shared --prefix=/usr/local/mpich $ make $ make install
Open MPI
$ tar -zxf openmpi-X.X.X tar.gz $ cd openmpi-X.X.X $ ./configure --prefix=/usr/local/openmpi $ make all $ make install
LAM/MPI
$ tar -zxf lam-X.X.X.tar.gz $ cd lam-X.X.X $ ./configure --enable-shared --prefix=/usr/local/lam $ make $ make install
MPICH 1
$ tar -zxf mpich-X.X.X.tar.gz $ cd mpich-X.X.X $ ./configure --enable-sharedlib --prefix=/usr/local/mpich1 $ make $ make install
Perhaps you will need to set the LD_LIBRARY_PATH
environment variable (using export, setenv or
what applies to your system) pointing to the directory containing the
MPI libraries . In case of getting runtime linking errors when running
MPI programs, the following lines can be added to the user login shell
script (.profile
, .bashrc
, etc.).
MPICH
MPI_DIR=/usr/local/mpich export LD_LIBRARY_PATH=$MPI_DIR/lib:$LD_LIBRARY_PATH
Open MPI
MPI_DIR=/usr/local/openmpi export LD_LIBRARY_PATH=$MPI_DIR/lib:$LD_LIBRARY_PATH
LAM/MPI
MPI_DIR=/usr/local/lam export LD_LIBRARY_PATH=$MPI_DIR/lib:$LD_LIBRARY_PATH
MPICH 1
MPI_DIR=/usr/local/mpich1 export LD_LIBRARY_PATH=$MPI_DIR/lib/shared:$LD_LIBRARY_PATH: export MPICH_USE_SHLIB=yes
Warning
MPICH 1 support for dynamic libraries is not completely transparent. Users should set the environment variable
MPICH_USE_SHLIB
toyes
in order to avoid link problems when using the mpicc compiler wrapper.