===== Installing Python 2.7.3 from Source ===== This guide follows from GCC 4.7.1 install from source Guide, and assumes that your bash environmental are setup accordingly. Building directory $ mkdir -p /tmp/$USER/python_build All libaries and binaries installed in $HOME/local folder, and will require and additional 110M of disk space. ===== Opptional pre-requisites ===== ==== zlib ==== [[ http://zlib.net | zlib ]] is designed to be a free, general-purpose, legally unencumbered -- that is, not covered by any patents -- lossless data-compression library for use on virtually any computer hardware and operating system. $ cd /tmp/$USER/python_build $ wget http://zlib.net/zlib-1.2.7.tar.gz $ tar -xf zlib-1.2.7.tar.gz $ cd zlib-1.2.7 $ ./configure --prefix=$HOME/local --libdir=$HOME/local/lib64 --64 $ make $ make check $ make install ==== tcl/tk ==== [[http://www.tcl.tk/about/ | tcl/tk ]] installation. first tcl $ cd /tmp/$USER/python_build $ wget http://prdownloads.sourceforge.net/tcl/tcl8.5.12-src.tar.gz $ tar -xf tcl8.5.12-src.tar.gz $ cd tcl8.5.12 $ mkdir BUILD $ cd BUILD $ ../unix/configure --prefix=$HOME/local --libdir=$HOME/local/lib64 \ --enable-64bit --enable-64bit-vis $ make $ make test Summary of test results were Tests ended at Mon Sep 10 16:00:00 SAST 2012 all.tcl: Total 27048 Passed 25880 Skipped 1166 Failed 2 Sourced 137 Test Files. Files with failing tests: httpold.test unixInit.test If acceptable, proceed $ make install then tk $ cd /tmp/$USER/python_build $ wget http://prdownloads.sourceforge.net/tcl/tk8.5.12-src.tar.gz $ tar -xf tk8.5.12-src.tar.gz $ mkdir tk8.5.12/BUILD $ cd tk8.5.12/BUILD $ ../unix/configure --with-x --prefix=$HOME/local --libdir=$HOME/local/lib64 \ --enable-64bit --enable-64bit-vis $ make $ make install ==== sqlite 3 ==== [[ http://www.sqlite.org/about.html | SQLite ]] is a in-process library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. $ cd /tmp/$USER/python_build $ wget http://www.sqlite.org/sqlite-autoconf-3071400.tar.gz $ tar -xf sqlite-autoconf-3071400.tar.gz $ cd sqlite-autoconf-3071400 $ mkdir BUILD $ cd BUILD $ ../configure --prefix=$HOME/local --libdir=$HOME/local/lib64 $ make no tests with package? $ make install Got warnings ldconfig: $HOME/local/lib64/libisl.so.10.0.0-gdb.py is not an ELF file - it has the wrong magic bytes at the start. Where ELF = Executable and Linkable Format (ELF, formerly called Extensible Linking Format). This is assumed okay as these are python module files, and not shared object files. The .so.10.0.0-gdb.py extention is probabily confusing ldconfig, as least i hope. Online docs also indicate that reconfiguring ld may fix the problem. If there is problem, try reinstalling binutils? ==== GDB : the GNU debugger ==== $ download_and_install_from_source.sh \ http://ftp.gnu.org/gnu/gdb/gdb-7.5.tar.bz2 \ /tmp/$USER/python_build "--libdir=$HOME/local/lib64 --with-python" $HOME/local Testing took 20 minutes to complete Got quite a lot of failures here... FIXME please... ===== Download and Configuration ===== $ cd /tmp/$USER/python_build $ wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2 $ tar -xf Python-2.7.3.tar.bz2 $ mkdir Python-2.7.3/BUILD $ cd Python-2.7.3/BUILD $ ../configure --prefix=$HOME/local Does thread support work? if not add the --with-threads=no option to the above configure ===== Compile ===== $ screen $ make > make.out 2>&1 # ctrl-a ctrl-d to detach screen $ tail -f make.out # to watch compilation progress giving Python build finished, but the necessary bits to build these modules were not found: bsddb185 dl imageop sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. Where * dl is depreciated since Python 2.6, use ctypes instead. * Beginning in 2.3 some Unix versions of Python may have a bsddb185 module. This is present only to allow backwards compatibility with systems which ship with the old Berkeley DB 1.85 database library. The bsddb185 module should never be used directly in new code. The module has been removed in Python 3. If you find you still need it look in PyPI. * imageop and sunaudiodev, do not work on 64 bit systems. ===== Test ===== $ screen $ make test > make.test 2>&1 # ctrl-a ctrl-d to detach screen $ tail -f make.test Test Summary: 352 tests OK. 1 test failed: test_gdb 36 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dl test_gl test_imageop test_imgfile test_kqueue test_linuxaudiodev test_macos test_macostools test_msilib test_ossaudiodev test_scriptpackages test_smtpnet test_socketserver test_startfile test_sunaudiodev test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 2 skips unexpected on linux2: test_tk test_ttk_guionly The unexpected test were skipped -- tk not available: no display name and no $DISPLAY environment variable As for test_gdb, which i assume is the GNU debugger interfacing module, more information on the error can be obtained by $ ./python ../Lib/test/regrtest.py -v test_gdb If this is good enough for your applications proceed with the installation. ===== Installation ===== $ make install $ ln -s ~/local/lib/libpython2.7.a ~/local/lib64/ clean up after yourself, $ rm -r /tmp/$USER/python_build Then you should see the following $ python Python 2.7.3 (default, Sep 11 2012, 11:01:22) [GCC 4.7.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> ===== Installing Some commom python packages ===== ==== setupTools ==== Start with python setup tools, as it contains the easy install tool. $ cd ~/scratch $ wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz $ tar -xf setuptools-0.6c11.tar.gz $ cd setuptools-0.6c11 $ python setup.py install Refer to the [[http://peak.telecommunity.com/DevCenter/EasyInstall|reference and manual website]] for further information on using easy install. ==== Nose ==== Nose is a testing module, used by packages such as Numpy and Scipy. To install simply $ easy_install nose ==== iPython ==== An enhanced Interactive Python shell, with lots of goodies ... $ easy_install ipython ==== mpi4py ==== [[https://code.google.com/p/mpi4py/|mpi4py]] is python interface to [[http://www.open-mpi.org/|openmpi]] which is a High Performance Message Passing Library. To start we are going to build and test the openmpi libraries. $ cd ~/scratch $ wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.1.tar.bz2 $ tar -xf openmpi-1.6.1.tar.bz2 $ cd openmpi-1.6.1 $ mkdir BUILD $ cd BUILD $ ../configure --prefix=$HOME/local --libdir=$HOME/local/lib64 configure taks a minutes or two, with lots of output .... $ make $ make check > make.check 2>&1 #takes a minute to complete $ grep -i -e pass -e fail make.check $ make install Download mpi4py $ cd ~/scratch $ easy_install --editable --build-directory . mpi4py $ cd mpi4py $ python setup.py build $ python setup.py test if no errors install $ python setup.py install ==== NumPy and SciPy==== [[http://numpy.scipy.org/|NumPy]] is the fundamental package for scientific computing with Python. It contains among other things: * a powerful N-dimensional array object * sophisticated (broadcasting) functions * tools for integrating C/C++ and Fortran code * useful linear algebra, Fourier transform, and random number capabilities Based on [[http://www.scipy.org/Installing_SciPy/Linux|installation guide]]. === BLAS and LAPACK libraries === Following this [[http://www.tfinley.net/notes/blas-lapack/|guide]] Download the LAPACK source and extract, setup make and build as follows, $ cd ~/scratch $ wget http://www.netlib.org/lapack/lapack-3.4.1.tgz $ tar -xf lapack-3.4.1.tgz $ cd lapack-3.4.1 Setup make, $ cp make.inc.example make.inc $ emacs make.inc # change # OPTS = -O2 and, NOOPT = -O0 # to # OPTS = -O2 -fPIC and, NOOPT = -O0 -fPIC # save (crtl-x crtl-s) and exit (crtl-x ctrl-c) $ make blaslib Make LAPACK $ screen $ make all > make.out 2>&1 # crtl-a d $ tail -f make.out After 10 minutes make will finish, and you should see test results such as --> LAPACK TESTING SUMMARY <-- Processing LAPACK Testing output found in the TESTING direcory SUMMARY nb test run numerical error other error REAL 1077227 0 (0.000%) 0 (0.000%) DOUBLE PRECISION 1078039 0 (0.000%) 0 (0.000%) COMPLEX 522814 0 (0.000%) 0 (0.000%) COMPLEX16 552410 0 (0.000%) 0 (0.000%) --> ALL PRECISIONS 3230490 0 (0.000%) 0 (0.000%) if all went well $ cp -iv lib* $HOME/local/lib64 To convert the built static libraries to dynamic libraries, the following bash loop is used $ for LIB in *.a ; do DIR=${LIB}_contents SOLIB=${LIB%.*}.so echo $SOLIB mkdir $DIR cd $DIR ar -x ../$LIB cd - gfortran -shared -Wl,-soname=$SOLIB -o $SOLIB ${DIR}/*.o done; $ ln -snf librefblas.so libblas.so Copy the shared object or dynamic libraries to your library folder $ cp -iv lib*.so $HOME/local/lib64 [warning dynamic libaries not fully tested] === Numpy building, testing and installation === Download numpy $ cd ~/scratch $ wget http://tenet.dl.sourceforge.net/project/numpy/NumPy/1.6.2/numpy-1.6.2.tar.gz $ tar -xf numpy-1.6.2.tar.gz $ cd numpy-1.6.2 == build using shared object libraries == $ echo "[DEFAULT]" > site.cfg $ echo "library_dirs = $HOME/local/lib64" >> site.cfg $ echo "include_dirs = $HOME/local/include" >> site.cfg $ python setup.py build --fcompiler=gnu95 $ python setup.py install == build using static lapack and blas libraries (for interests sake only) == Set blas and lapack paths $ export ATLAS=None $ export BLAS=$HOME/local/lib64/librefblas.a $ export LAPACK=$HOME/local/lib64/liblapack.a Build and install $ python setup.py build --fcompiler=gnu95 NB to build without lapack and blas libraries, do the following $ export ATLAS=None $ export BLAS=None $ export LAPACK=None then build speed difference: using LAPACK and BLAS libraries numpy testing takes between 10 and 12s, without the libraries numpy.test() takes exactly the same amount of time? == testing == $ mkdir -p /tmp/$USER/numpy-1.6.2_testing $ python setup.py install --prefix=/tmp/$USER/numpy-1.6.2_testing $ ipython $ import os, sys $ sys.path.insert(0,'/tmp/%s/numpy-1.6.2_testing/lib/python2.7/site-packages' % os.environ['USER']) $ import numpy $ numpy.test() If all the test were passed, rm the testing directory and install numpy $ rm -r /tmp/$USER/numpy-1.6.2_testing $ python setup.py install === Scipy, minimal install === $ cd ~/scratch $ wget http://tenet.dl.sourceforge.net/project/scipy/scipy/0.10.1/scipy-0.10.1.tar.gz $ tar -xf scipy-0.10.1.tar.gz $ cd scipy-0.10.1/ $ python setup.py build #takes about 10 minutes to build testing, $ mkdir -p /tmp/$USER/scipy-0.10.1 $ python setup.py install --prefix=/tmp/$USER/scipy-0.10.1_testing $ ipython $ import os, sys $ sys.path.insert(0,'/tmp/%s/scipy-0.10.1_testing/lib/python2.7/site-packages' % os.environ['USER']) $ import scipy $ scipy.test() which did not work 100 % :( FAILED (KNOWNFAIL=13, SKIP=41, failures=1) Out[2]: If testing satisfactory, rm the testing directory and install scipy $ rm -r /tmp/$USER/scipy-0.10.1_testing $ python setup.py install ==== matplotlib / pylab ==== Download latest version from source forge (version 1.1 in this case) $ cd $MATPLOTLIB_FOLDER $ python setup.py install