Table of Contents

Installing Python 2.7.3 from Source

This guide follows from GCC 4.7.1 install from source Guide, and assumes that your bash environmental are setup accordingly. Building directory

$ mkdir -p /tmp/$USER/python_build

All libaries and binaries installed in $HOME/local folder, and will require and additional 110M of disk space.

Opptional pre-requisites

zlib

zlib is designed to be a free, general-purpose, legally unencumbered – that is, not covered by any patents – lossless data-compression library for use on virtually any computer hardware and operating system.

$ cd /tmp/$USER/python_build
$ wget http://zlib.net/zlib-1.2.7.tar.gz
$ tar -xf zlib-1.2.7.tar.gz
$ cd zlib-1.2.7
$ ./configure --prefix=$HOME/local --libdir=$HOME/local/lib64 --64
$ make
$ make check
$ make install 

tcl/tk

tcl/tk installation. first tcl

$ cd /tmp/$USER/python_build
$ wget http://prdownloads.sourceforge.net/tcl/tcl8.5.12-src.tar.gz
$ tar -xf tcl8.5.12-src.tar.gz
$ cd tcl8.5.12
$ mkdir BUILD
$ cd BUILD
$ ../unix/configure --prefix=$HOME/local --libdir=$HOME/local/lib64 \
--enable-64bit --enable-64bit-vis 
$ make
$ make test

Summary of test results were

Tests ended at Mon Sep 10 16:00:00 SAST 2012
all.tcl:    Total  27048  Passed   25880	Skipped	1166	Failed	2
Sourced 137 Test Files.
Files with failing tests: httpold.test unixInit.test

If acceptable, proceed

$ make install

then tk

$ cd /tmp/$USER/python_build
$ wget http://prdownloads.sourceforge.net/tcl/tk8.5.12-src.tar.gz
$ tar -xf tk8.5.12-src.tar.gz
$ mkdir tk8.5.12/BUILD
$ cd tk8.5.12/BUILD
$ ../unix/configure --with-x --prefix=$HOME/local --libdir=$HOME/local/lib64  \
--enable-64bit --enable-64bit-vis
$ make
$ make install

sqlite 3

SQLite is a in-process library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine.

$ cd /tmp/$USER/python_build
$ wget http://www.sqlite.org/sqlite-autoconf-3071400.tar.gz
$ tar -xf sqlite-autoconf-3071400.tar.gz
$ cd sqlite-autoconf-3071400
$ mkdir BUILD
$ cd BUILD
$ ../configure --prefix=$HOME/local --libdir=$HOME/local/lib64 
$ make

no tests with package?

$ make install

Got warnings ldconfig: $HOME/local/lib64/libisl.so.10.0.0-gdb.py is not an ELF file - it has the wrong magic bytes at the start.

Where ELF = Executable and Linkable Format (ELF, formerly called Extensible Linking Format). This is assumed okay as these are python module files, and not shared object files. The .so.10.0.0-gdb.py extention is probabily confusing ldconfig, as least i hope.

Online docs also indicate that reconfiguring ld may fix the problem. If there is problem, try reinstalling binutils?

GDB : the GNU debugger

$ download_and_install_from_source.sh \
http://ftp.gnu.org/gnu/gdb/gdb-7.5.tar.bz2 \
/tmp/$USER/python_build "--libdir=$HOME/local/lib64 --with-python" $HOME/local

Testing took 20 minutes to complete Got quite a lot of failures here…

FIXME please…

Download and Configuration

$ cd /tmp/$USER/python_build
$ wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
$ tar -xf Python-2.7.3.tar.bz2
$ mkdir Python-2.7.3/BUILD
$ cd Python-2.7.3/BUILD
$ ../configure --prefix=$HOME/local 

Does thread support work? if not add the –with-threads=no option to the above configure

Compile

$ screen 
$ make > make.out 2>&1
# ctrl-a ctrl-d to detach screen
$ tail -f make.out # to watch compilation progress

giving

Python build finished, but the necessary bits to build these modules were not
found:
bsddb185           dl        imageop
sunaudiodev   
To find the necessary bits, look in setup.py in detect_modules() for the
module's name.

Where

Test

$ screen
$ make test > make.test 2>&1
# ctrl-a ctrl-d to detach screen
$ tail -f make.test

Test Summary:

352 tests OK.
1 test failed:
   test_gdb
36 tests skipped:
   test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
   test_cd test_cl test_codecmaps_cn test_codecmaps_hk
   test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
   test_dl test_gl test_imageop test_imgfile test_kqueue
   test_linuxaudiodev test_macos test_macostools test_msilib
   test_ossaudiodev test_scriptpackages test_smtpnet
   test_socketserver test_startfile test_sunaudiodev test_timeout
   test_tk test_ttk_guionly test_urllib2net test_urllibnet
   test_winreg test_winsound test_zipfile64
2 skips unexpected on linux2:
   test_tk test_ttk_guionly

The unexpected test were

skipped -- tk not available: no display name and no $DISPLAY environment variable

As for test_gdb, which i assume is the GNU debugger interfacing module, more information on the error can be obtained by

$ ./python ../Lib/test/regrtest.py -v test_gdb

If this is good enough for your applications proceed with the installation.

Installation

$ make install
$ ln -s ~/local/lib/libpython2.7.a ~/local/lib64/

clean up after yourself,

$ rm -r /tmp/$USER/python_build

Then you should see the following

$ python
Python 2.7.3 (default, Sep 11 2012, 11:01:22) 
[GCC 4.7.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> 

Installing Some commom python packages

setupTools

Start with python setup tools, as it contains the easy install tool.

$ cd ~/scratch
$ wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz
$ tar -xf setuptools-0.6c11.tar.gz
$ cd setuptools-0.6c11
$ python setup.py install

Refer to the reference and manual website for further information on using easy install.

Nose

Nose is a testing module, used by packages such as Numpy and Scipy. To install simply

$ easy_install nose

iPython

An enhanced Interactive Python shell, with lots of goodies …

$ easy_install ipython

mpi4py

mpi4py is python interface to openmpi which is a High Performance Message Passing Library.

To start we are going to build and test the openmpi libraries.

$ cd ~/scratch
$ wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.1.tar.bz2
$ tar -xf openmpi-1.6.1.tar.bz2
$ cd openmpi-1.6.1
$ mkdir BUILD
$ cd BUILD
$ ../configure --prefix=$HOME/local --libdir=$HOME/local/lib64

configure taks a minutes or two, with lots of output ….

$ make
$ make check > make.check 2>&1  #takes a minute to complete
$ grep -i -e pass -e fail make.check
$ make install

Download mpi4py

$ cd ~/scratch
$ easy_install --editable --build-directory . mpi4py
$ cd mpi4py
$ python setup.py build 
$ python setup.py test

if no errors install

$ python setup.py install

NumPy and SciPy

NumPy is the fundamental package for scientific computing with Python. It contains among other things:

Based on installation guide.

BLAS and LAPACK libraries

Following this guide

Download the LAPACK source and extract, setup make and build as follows,

$ cd ~/scratch
$ wget http://www.netlib.org/lapack/lapack-3.4.1.tgz
$ tar -xf lapack-3.4.1.tgz
$ cd lapack-3.4.1

Setup make,

$ cp make.inc.example make.inc
$ emacs make.inc
# change 
#  OPTS     = -O2  and, NOOPT    = -O0
# to
#  OPTS     = -O2 -fPIC and, NOOPT    = -O0 -fPIC
# save (crtl-x crtl-s) and exit (crtl-x ctrl-c)
$ make blaslib

Make LAPACK

$ screen
$ make all > make.out 2>&1
# crtl-a d
$ tail -f make.out

After 10 minutes make will finish, and you should see test results such as

	 -->   LAPACK TESTING SUMMARY  <--
     Processing LAPACK Testing output found in the TESTING  direcory
SUMMARY		       nb test run	numerical error		other error  
REAL		        1077227		0	(0.000%)	0   (0.000%)	
DOUBLE PRECISION	1078039		0	(0.000%)	0   (0.000%)	
COMPLEX	                522814		0	(0.000%)	0   (0.000%)	
COMPLEX16		552410		0	(0.000%)	0   (0.000%)	
--> ALL PRECISIONS	3230490		0	(0.000%)	0   (0.000%)

if all went well

$ cp -iv lib* $HOME/local/lib64

To convert the built static libraries to dynamic libraries, the following bash loop is used

$ for LIB in *.a ; do
    DIR=${LIB}_contents
    SOLIB=${LIB%.*}.so
    echo $SOLIB
    mkdir $DIR
    cd $DIR
    ar -x ../$LIB
    cd -
    gfortran -shared -Wl,-soname=$SOLIB -o $SOLIB ${DIR}/*.o
  done;
$ ln -snf librefblas.so libblas.so

Copy the shared object or dynamic libraries to your library folder

$ cp -iv lib*.so $HOME/local/lib64

[warning dynamic libaries not fully tested]

Numpy building, testing and installation

Download numpy

$ cd ~/scratch
$ wget   http://tenet.dl.sourceforge.net/project/numpy/NumPy/1.6.2/numpy-1.6.2.tar.gz
$ tar -xf numpy-1.6.2.tar.gz
$ cd numpy-1.6.2
build using shared object libraries
$ echo "[DEFAULT]" > site.cfg
$ echo "library_dirs = $HOME/local/lib64" >> site.cfg
$ echo "include_dirs = $HOME/local/include"  >> site.cfg
$ python setup.py build --fcompiler=gnu95
$ python setup.py install
build using static lapack and blas libraries (for interests sake only)

Set blas and lapack paths

$ export ATLAS=None
$ export BLAS=$HOME/local/lib64/librefblas.a
$ export LAPACK=$HOME/local/lib64/liblapack.a

Build and install

$ python setup.py build --fcompiler=gnu95

NB to build without lapack and blas libraries, do the following

$ export ATLAS=None
$ export BLAS=None
$ export LAPACK=None

then build

speed difference: using LAPACK and BLAS libraries numpy testing takes between 10 and 12s, without the libraries numpy.test() takes exactly the same amount of time?

testing
$ mkdir -p /tmp/$USER/numpy-1.6.2_testing
$ python setup.py install --prefix=/tmp/$USER/numpy-1.6.2_testing
$ ipython
  $ import os, sys
  $ sys.path.insert(0,'/tmp/%s/numpy-1.6.2_testing/lib/python2.7/site-packages'  % os.environ['USER'])
  $ import numpy
  $ numpy.test()

If all the test were passed, rm the testing directory and install numpy

$ rm -r /tmp/$USER/numpy-1.6.2_testing
$ python setup.py install

Scipy, minimal install

$ cd ~/scratch
$ wget http://tenet.dl.sourceforge.net/project/scipy/scipy/0.10.1/scipy-0.10.1.tar.gz
$ tar -xf scipy-0.10.1.tar.gz
$ cd scipy-0.10.1/
$ python setup.py build #takes about 10 minutes to build

testing,

$ mkdir -p /tmp/$USER/scipy-0.10.1
$ python setup.py install --prefix=/tmp/$USER/scipy-0.10.1_testing
$ ipython
  $ import os, sys
  $ sys.path.insert(0,'/tmp/%s/scipy-0.10.1_testing/lib/python2.7/site-packages'  % os.environ['USER'])
  $ import scipy
  $ scipy.test()

which did not work 100 % :(

FAILED (KNOWNFAIL=13, SKIP=41, failures=1)
Out[2]: <nose.result.TextTestResult run=5101 errors=0 failures=1>

If testing satisfactory, rm the testing directory and install scipy

$ rm -r /tmp/$USER/scipy-0.10.1_testing
$ python setup.py install

matplotlib / pylab

Download latest version from source forge (version 1.1 in this case)
$  cd $MATPLOTLIB_FOLDER
$  python setup.py install