This is an old revision of the document!
This guide follows from GCC 4.7.1 install from source Guide, and assumes that your bash environmental are setup accordingly. Building directory
$ mkdir -p /tmp/$USER/python_build
All libaries and binaries installed in $HOME/usr folder, and will require and additional 110M of disk space.
zlib is designed to be a free, general-purpose, legally unencumbered – that is, not covered by any patents – lossless data-compression library for use on virtually any computer hardware and operating system.
$ cd /tmp/$USER/python_build $ wget http://zlib.net/zlib-1.2.7.tar.gz $ tar -xf zlib-1.2.7.tar.gz $ cd zlib-1.2.7 $ ./configure --prefix=$HOME/local --libdir=$HOME/local/lib64 --64 $ make $ make check $ make install
tcl/tk installation. first tcl
$ cd /tmp/$USER/python_build $ wget http://prdownloads.sourceforge.net/tcl/tcl8.5.12-src.tar.gz $ tar -xf tcl8.5.12-src.tar.gz $ cd tcl8.5.12 $ mkdir BUILD $ cd BUILD $ ../unix/configure --prefix=$HOME/local --libdir=$HOME/local/lib64 \ --enable-64bit --enable-64bit-vis $ make $ make test
Summary of test results were
Tests ended at Mon Sep 10 16:00:00 SAST 2012 all.tcl: Total 27048 Passed 25880 Skipped 1166 Failed 2 Sourced 137 Test Files. Files with failing tests: httpold.test unixInit.test
If acceptable, proceed
$ make install
then tk
$ cd /tmp/$USER/python_build $ wget http://prdownloads.sourceforge.net/tcl/tk8.5.12-src.tar.gz $ tar -xf tk8.5.12-src.tar.gz $ mkdir tk8.5.12/BUILD $ cd tk8.5.12/BUILD $ ../unix/configure --with-x --prefix=$HOME/local --libdir=$HOME/local/lib64 \ --enable-64bit --enable-64bit-vis $ make $ make install
SQLite is a in-process library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine.
$ cd /tmp/$USER/python_build $ wget http://www.sqlite.org/sqlite-autoconf-3071400.tar.gz $ tar -xf sqlite-autoconf-3071400.tar.gz $ cd sqlite-autoconf-3071400 $ mkdir BUILD $ cd BUILD $ ../configure --prefix=$HOME/local --libdir=$HOME/local/lib64 $ make
no tests with package?
$ make install
Got warnings ldconfig: $HOME/local/lib64/libisl.so.10.0.0-gdb.py is not an ELF file - it has the wrong magic bytes at the start.
Where ELF = Executable and Linkable Format (ELF, formerly called Extensible Linking Format). This is assumed okay as these are python module files, and not shared object files. The .so.10.0.0-gdb.py extention is probabily confusing ldconfig, as least i hope.
Online docs also indicate that reconfiguring ld may fix the problem. If there is problem, try reinstalling binutils?
$ download_and_install_from_source.sh \ http://ftp.gnu.org/gnu/gdb/gdb-7.5.tar.bz2 \ /tmp/$USER/python_build "--libdir=$HOME/local/lib64 --with-python" $HOME/local
Testing took 20 minutes to complete Got quite a lot of failures here…
#
# # http://www.oracle.com/technology/software/products/berkeley-db/index.html # #After reading Python-2.7.3/Modules/_bsddb.c # #from Python-2.7.3/Modules/_bsddb.c # $ cd /tmp/$USER/python_build # $ wget http://download.oracle.com/berkeley-db/db-4.2.52.tar.gz # $ tar -xf db-4.2.52.tar.gz # $ cd db-4.2.52/build_unix #For some reason the configure script does not scan the BASH's PATH path #enviromental variable so, # $ export CC=gcc # $ export CXX=g++ # $ export CXXFLAGS=“-I/export/home/adymond/usr/include/c++/4.7.1 $CPPFLAGS” # $ ../dist/configure –with-tcl=$HOME/usr/lib –enable-test –enable-cxx –prefix=$HOME/usr # $ make #Test using the tclsh # $ tclsh # % source ../test/test.tcl # % run_std # # ctrl-d to exit # $ make install could not compile, my theory is that version 4.2 is old and not compadible with the g++ 4.7.1 compiler. This theory is support by the fact the version 5.1.29 compiles fine (and passes all its test?), using exactly the same #settings.
#setting back to gcc 4.1.2 and trying again # $ unset CC CFLAGS LDFLAGS CPPFLAGS CXX CXXFLAGS CXXCPP CPP F77 FFLAGS # $ export CC=gcc #else /usr/bin/cc is used # $ export PATH=/opt/gridware/moab/7.1.1/sbin:/opt/gridware/moab/7.1.1/bin:/opt/gridware/torque/2.5.12/sbin:/opt/gridware/torque/2.5.12/bin:/opt/gridware/moab/7.1.1/sbin:/opt/gridware/moab/7.1.1/bin:/opt/gridware/torque/2.5.12/sbin:/opt/gridware/torque/2.5.12/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/opt/gridware/moab/7.1.1/bin:/opt/gridware/moab/7.1.1/sbin:/opt/gridware/mam-7.1/bin:/opt/gridware/mam-7.1/sbin:/opt/gridware/mongodb-linux-x86_64-static-legacy-2.0.7/bin:/opt/gridware/moab/7.1.1/bin:/opt/gridware/moab/7.1.1/sbin:/opt/gridware/mam-7.1/bin:/opt/gridware/mam-7.1/sbin:/opt/gridware/mongodb-linux-x86_64-static-legacy-2.0.7/bin # $ ../dist/configure –with-tcl=$HOME/usr/lib –enable-test –enable-cxx #–prefix=$HOME/usr #also does not work …
#Q: _bsddb module compiles with out this, are there libaries in /usr/lib which #are being used during the python build? #A: yes /usr/lib has #/usr/lib/libdb-4.3.a /usr/lib/libdb-4.3.so /usr/lib/libdb_cxx-4.3.la #/usr/lib/libdb_cxx.so /usr/lib/libdb-4.3.la /usr/lib/libdb_cxx-4.3.a #/usr/lib/libdb_cxx-4.3.so /usr/lib/libdb.so
#so, what happened here. Did in the past when i build without the #'–enable-cxx' flag, we the python build trying to link to these #/usr/lib/libdb_cxx libraries? # using the /usr/lib libs on there own with the tcl and tk just installed #seems to work :)
$ cd /tmp/$USER/python_build $ wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2 $ tar -xf Python-2.7.3.tar.bz2 $ mkdir Python-2.7.3/BUILD $ cd Python-2.7.3/BUILD $ ../configure --prefix=$HOME/local
Does thread support work? if not add the –with-threads=no option to the above configure
$ screen $ make > make.out 2>&1 # ctrl-a ctrl-d to detach screen $ tail -f make.out # to watch compilation progress
giving
Python build finished, but the necessary bits to build these modules were not found: bsddb185 dl imageop sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Where
module. This is present only to allow backwards compatibility with systems
which ship with the old Berkeley DB 1.85 database library. The bsddb185 module should never be * imageop and sunaudiodev, do not work on 64 bit systems.
used directly in new code. The module has been removed in Python 3. If you find you still need it look in PyPI.
$ screen $ make test > make.test 2>&1 # ctrl-a ctrl-d to detach screen $ tail -f make.test
Test Summary:
352 tests OK. 1 test failed: test_gdb 36 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dl test_gl test_imageop test_imgfile test_kqueue test_linuxaudiodev test_macos test_macostools test_msilib test_ossaudiodev test_scriptpackages test_smtpnet test_socketserver test_startfile test_sunaudiodev test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 2 skips unexpected on linux2: test_tk test_ttk_guionly
The unexpected test were
skipped -- tk not available: no display name and no $DISPLAY environment variable
As for test_gdb, which i assume is the GNU debugger interfacing module, more information on the error can be obtained by ./python ../Lib/test/regrtest.py -v test_gdb
If this is good enough for your applications proceed with the installation.
$ make install $ ln -s ~/local/lib/libpython2.7.a ~/local/lib64/
clean up after yourself,
$ rm -r /tmp/$USER/python_build
Then you should see the following
$ python Python 2.7.3 (default, Sep 11 2012, 11:01:22) [GCC 4.7.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
Start with python setup tools, as it contains the easy install tool.
$ cd ~/scratch $ wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz $ tar -xf setuptools-0.6c11.tar.gz $ cd setuptools-0.6c11 $ python setup.py install
Refer to the reference and manual website for further information on using easy install.
Nose is a testing module, used by packages such as Numpy and Scipy. To install simply
$ easy_install nose
An enhanced Interactive Python shell, with lots of goodies …
$ easy_install ipython
mpi4py is python interface to openmpi which is a High Performance Message Passing Library.
To start we are going to build and test the openmpi libraries.
$ cd ~/scratch $ wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.1.tar.bz2 $ tar -xf openmpi-1.6.1.tar.bz2 $ cd openmpi-1.6.1 $ mkdir BUILD $ cd BUILD $ ../configure --prefix=$HOME/local --libdir=$HOME/local/lib64
configure taks a minutes or two, with lots of output ….
$ make $ make check > make.check 2>&1 #takes a minute to complete $ grep -i -e pass -e fail make.check $ make install
Download mpi4py
$ cd ~/scratch $ easy_install --editable --build-directory . mpi4py $ cd mpi4py $ python setup.py build $ python setup.py test
if no errors install
$ python setup.py install
NumPy is the fundamental package for scientific computing with Python. It contains among other things:
Based on installation guide.
==== Build ATLAS(3.9.32) with complete Lapack(3.2.2)
From a temporary director ( ie Scratch ) download the latest atlas from Source forge
$ wget http://www.netlib.org/lapack/lapack-3.4.0.tgz $ wget http://tenet.dl.sourceforge.net/project/math-atlas/Stable/3.10.0/atlas3.10.0.tar.bz2 $ tar -xf atlas3.10.0.tar.bz2 $ mkdir ATLAS/BUILD $ cd ATLAS/BUILD $ ../configure -b 64 -Fa alg -fPIC --shared \ --with-netlib-lapack-tarfile=../../lapack-3.4.0.tgz --prefix=$HOME/usr
make cd lib make shared make ptshared cd .. make install
Following this guide
Download the LAPACK source and extract, setup make and build as follows,
$ cd ~/scratch $ wget http://www.netlib.org/lapack/lapack-3.4.1.tgz $ tar -xf lapack-3.4.1.tgz $ cd lapack-3.4.1
Setup make,
$ cp make.inc.example make.inc $ emacs make.inc # change # OPTS = -O2 and, NOOPT = -O0 # to # OPTS = -O2 -fPIC and, NOOPT = -O0 -fPIC # save (crtl-x crtl-s) and exit (crtl-x ctrl-c) $ make blaslib
Make LAPACK
$ screen $ make all > make.out 2>&1 # crtl-a d $ tail -f make.out
After 10 minutes make will finish, and you should see test results such as
--> LAPACK TESTING SUMMARY <--
Processing LAPACK Testing output found in the TESTING direcory
SUMMARY nb test run numerical error other error
REAL 1077227 0 (0.000%) 0 (0.000%) DOUBLE PRECISION 1078039 0 (0.000%) 0 (0.000%) COMPLEX 522814 0 (0.000%) 0 (0.000%) COMPLEX16 552410 0 (0.000%) 0 (0.000%) –> ALL PRECISIONS 3230490 0 (0.000%) 0 (0.000%)
if all went well
$ cp -iv lib* $HOME/local/lib64
#Compile the shared object (so) or dynamic libraries for blas and lap#ack # $ gcc -shared BLAS/SRC/*.o -o libblas.so # $ gcc -shared SRC/*.o -o liblapack.so #Basic library checks, # $ ipython # $ from ctypes import * # $ blaslib = cdll.LoadLibrary('libblas.so') # $ array = (c_float*4)() # $ array[:] = [4.0, 3.14159, 4.1, 2.0] # $ print blaslib.isamax_(byref(c_int(len(array))), array, byref(c_int(1))) # The output should be 3. # $ lapacklib = cdll.LoadLibrary('liblapack.so') # $ n, info = 5, c_int() # $ diag, offdiag = (c_double*n)(), (c_double*(n-1))() # $ diag[:], offdiag[:] = [2.0]*n, [-1.0]*(n-1) # $ r=lapacklib.dsterf_(byref(c_int(n)), diag, offdiag, byref(info)) # $ print '[', ', '.join('%g'%v for v in diag[:]), ']' # You should get output like [ 0.267949, 1, 2, 3, 3.73205 ], the eigenvalues # for this special matrix.
#If so congratulations. Then copy the compiled libaries to your $HOME/usr #directory # $ cp lib* $HOME/usr/lib
#Download the last version of numpy source, in your scratch folder. # $ cd ~/scratch # $ easy_install –editable –build-directory . numpy # $ cd numpy #Or alternatively Download numpy
$ cd ~/scratch $ wget http://tenet.dl.sourceforge.net/project/numpy/NumPy/1.6.2/numpy-1.6.2.tar.gz $ tar -xf numpy-1.6.2.tar.gz $ cd numpy-1.6.2
#Before continuing read both INSTALL.txt and site.cfg.example # $ echo '[DEFAULT]' > site.cfg # $ echo “library_dirs = $HOME/usr/lib64:$HOME/usr/lib” » site.cfg # $ echo “include_dirs = $HOME/usr/include:$HOME/usr/include/python2.7” » site.cfg
#nun python setup.py install. # $ python setup.py build –fcompiler=gnu95 # $ python setup.py install #started at 09:25
#try unsetting CPPFLAGS , and setting include dirs in set.cfg, #maybe this will avoid #_configtest.c:1:5: warning: conflicting types for built-in function ‘exp’ #[enabled by default] #gcc -pthread _configtest.o -o _configtest #_configtest.o: In function `main': #/tmp/adymond/numpy-1.6.2/_configtest.c:6: undefined reference to `exp' #while -lm (linking to math library should fix that, how do i add that to gcc
# $ unset CPPFLAGS # $ export CPPFLAGS='-lm'
#Sidenote: To compile with BLAS and LAPACK, set the following variables before building # $ export ATLAS=None # $ export BLAS=None # $ export LAPACK=None
Set blas and lapack paths
$ export ATLAS=None $ export BLAS=$HOME/local/lib64/librefblas.a $ export LAPACK=$HOME/local/lib64/liblapack.a
Build and install
$ python setup.py build --fcompiler=gnu95 $ python setup.py install
Then test it (note this requires nose)
$ cd ~ $ ipython $ import numpy $ numpy.test()
$ cd ~/scratch $ wget http://tenet.dl.sourceforge.net/project/scipy/scipy/0.10.1/scipy-0.10.1.tar.gz $ tar -xf scipy-0.10.1.tar.gz $ cd scipy-0.10.1/
Set blas and lapack paths
$ export ATLAS=None $ export BLAS=$HOME/local/lib64/librefblas.a $ export LAPACK=$HOME/local/lib64/liblapack.a
Build it, which takes a while
$ python setup.py build
Install it
$ python setup.py install
Once the build has been successfully completed, its time to install and test scipy
$ python setup.py install $ cd ~ $ ipython $ import scipy $ scipy.test()
which did not work 100 % :(
FAILED (KNOWNFAIL=13, SKIP=41, failures=1) Out[2]: <nose.result.TextTestResult run=5101 errors=0 failures=1>
Download latest version from source forge (version 1.1 in this case) $ cd $MATPLOTLIB_FOLDER $ python setup.py install