User Tools

Site Tools




source .bashrc

.bashrc :

export PATH=$HOME/bin:$PATH

#gcc 4.7.2
module add gcc/4.7.2
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

# Python 2.6.5 on /opt/gridware
export PATH=/opt/gridware/python2.6.5/bin:$PATH
export LD_LIBRARY_PATH=/opt/gridware/python2.6.5/lib/:$LD_LIBRARY_PATH

Thats it. The /opt/gridware python comes with nose, ipython, and numpy. However numpy is a little broken :

$ ipython
$ import numpy
$ numpy.test()

Installing packages

Assuming your codes are to be run on the dell cluster, add the following lines to your .bashrc file

#if feature=dell check still needs to be added
export PYTHONPATH=~/local_dell/lib/python:~/local_dell/lib/python2.6/site-packages/
export PATH=$HOME/local_dell/bin:$PATH

create the required directories

$ mkdir -p $HOME/local_dell/lib/python2.6/site-packages/
$ mkdir -p $HOME/local_dell/bin


Use for tool for managing and install Python packages. Setuptool provides the easy_install command

$ cd tmp
$ wget
$ tar -xf setuptools-0.6c11.tar.gz
$ cd setuptools-0.6c11
$ python install --prefix=$HOME/local_dell


From the login node (which has http access) simply

$ easy_install --prefix=$HOME/local_dell argparse

numpy (minimal)

$ msub -I -l feature=dell
$ cd ~/tmp
$ ssh login01 wget
$ mv ../numpy-1.6.2.tar.gz ./
$ tar -xf numpy-1.6.2.tar.gz
$ cd numpy-1.6.2
$ export ATLAS=None
$ export BLAS=None
$ export LAPACK=None
$ python build --fcompiler=gnu95
$ python install --prefix=$HOME/local_dell

when testing you should get

$ ipython
  import numpy
Ran 3568 tests in 11.266s

Out[3]: <nose.result.TextTestResult run=3568 errors=0 failures=0>

scipy (minimal)

$ msub -I -l feature=dell
$ cd ~/tmp
$ ssh login01 wget
$ mv ../scipy-0.10.1.tar.gz ./
$ tar -xf scipy-0.10.1.tar.gz
$ cd scipy-0.10.1/
$ python build --fcompiler=gnu95 #takes about 10 minutes to build
$ python install --prefix=$HOME/local_dell


$ cd ~/
$ ipython
  import scipy

which did not work 100 % :(

FAILED (KNOWNFAIL=13, SKIP=41, failures=2) Out[2]: <nose.result.TextTestResult run=5101 errors=0 failures=1>

matplotlib / pylab

$ msub -I -l feature=dell
$ cd ~/tmp
$ ssh login01 wget
$ mv ../matplotlib-1.2.0.tar.gz ./
$ tar -xf matplotlib-1.2.0.tar.gz
$ cd matplotlib-1.2.0
$ python install --prefix=$HOME/local_dell


Is already installed, all that is needed is to load the openmpi module; add the following line to .bashrc

module add openmpi-1.6.1-intel

User Package installation with Anaconda

Modules for anaconda version 2 and 3 are installed on lengau.To load these modules, use the following command (either from the command-line in an interactive session, or in a script:

  module add chpc/python/anaconda/2


  module add chpc/python/anaconda/3

Using virtualenv with anaconda is not recommended. However, Anaconda does include a similar functionality, which allows users to create an anaconda virtual environment.

The following instructions to set up your virtual environment. YOU NEED ONLY DO THEM ONCE. Please replace USERNAME with your actual username wherever it occurs below.

1. log into

2. ssh chpclic1 Unlike the compute nodes, chpcviz1 and chpclic1 have a connection to the internet, which is needed here

3. module load chpc/python/anaconda/3

you can substitute chpc/anaconda/2 if you want python 2

4. conda init bash

5. conda create –prefix /home/USERNAME/myenv python=3.6 anaconda

change python=3.6 to python=2.7 if you're using anaconda/2

6. conda activate /home/USERNAME/myenv

You don't have to use “myenv”, just replace it consistently with whatever name you prefer for your virtual environment. “myenv” will work, though.

7. You can now install whatever packages you need, that are not installed already for instance, for rpy2, type:

 conda install rpy2

8. you can now type:

conda deactivate /home/USERNAME/myenv

to exit the virtual environment.

Now, once you've done this, you can log out, and ready your job for submission from the login node. You must do the following in your job-submit script: Put it all near the top of your script, but after the #PBS directives:

module add chpc/python/anaconda/3
conda activate /home/USERNAME/myenv

and at the end:

conda deactivate /home/USERNAME/myenv

Then submit your script as normal with qsub.

if you ever need to install new python packages, just repeat steps 1, 2, 3, 6, 7 and 8 from above. (omit 4 and 5).

/var/www/wiki/data/pages/howto/python/start.txt · Last modified: 2019/08/22 15:12 by agill