wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » howto » python
Trace:
howto:python:start

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
howto:python:start [2019/08/22 15:03]
agill [User Package installation with Anaconda] 		howto:python:start [2021/12/09 16:42] (current)
Line 1: Line 1:
 ====== Python ====== ====== Python ======
  
-.profile +See the [[guide:python|Python User Guide]].
-  source .bashrc +
- +
-.bashrc : +
- +
-  export PATH=$HOME/bin:$PATH +
-   +
-  #gcc 4.7.2 +
-  module add gcc/4.7.2 +
-  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib +
-   +
-  # Python 2.6.5 on /opt/gridware +
-  export PATH=/opt/gridware/python2.6.5/bin:$PATH +
-  export LD_LIBRARY_PATH=/opt/gridware/python2.6.5/lib/:$LD_LIBRARY_PATH +
- +
-Thats it. The /opt/gridware python comes with nose, ipython, and numpy. However numpy is a little broken : +
-  $ ipython +
-  $ import numpy +
-  $ numpy.test() +
- +
-===== Installing packages ===== +
- +
-Assuming your codes are to be run on the dell cluster, add the following lines to your .bashrc file +
-  #if feature=dell check still needs to be added +
-  export PYTHONPATH=~/local_dell/lib/python:~/local_dell/lib/python2.6/site-packages/ +
-  export PATH=$HOME/local_dell/bin:$PATH +
-   +
-create the required directories +
-  $ mkdir -p $HOME/local_dell/lib/python2.6/site-packages/ +
-  $ mkdir -p $HOME/local_dell/bin +
- +
- +
-==== setuptools ====  +
- +
-Use for tool for managing and install Python packages. Setuptool provides the //easy_install// command +
- +
-  $ cd tmp +
-  $ wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz +
-  $ tar -xf setuptools-0.6c11.tar.gz +
-  $ cd setuptools-0.6c11 +
-  $ python setup.py install --prefix=$HOME/local_dell +
- +
-==== argparse ==== +
- +
-From the login node (which has http access) simply +
-  $ easy_install --prefix=$HOME/local_dell argparse +
- +
- +
-==== numpy (minimal) ==== +
- +
-  $ msub -I -l feature=dell +
-  $ cd ~/tmp +
-  $ ssh login01 wget http://tenet.dl.sourceforge.net/project/numpy/NumPy/1.6.2/numpy-1.6.2.tar.gz +
-  $ mv ../numpy-1.6.2.tar.gz ./ +
-  $ tar -xf numpy-1.6.2.tar.gz +
-  $ cd numpy-1.6.2 +
-  $ export ATLAS=None +
-  $ export BLAS=None +
-  $ export LAPACK=None +
-  $ python setup.py build --fcompiler=gnu95 +
-  $ python setup.py install --prefix=$HOME/local_dell +
- +
-when testing you should get  +
-  $ ipython +
-    import numpy +
-    numpy.test() +
-  Ran 3568 tests in 11.266s +
-   +
-  OK (KNOWNFAIL=5, SKIP=1) +
-  Out[3]: <nose.result.TextTestResult run=3568 errors=0 failures=0> +
- +
-==== scipy (minimal) ==== +
- +
- +
-  $ msub -I -l feature=dell +
-  $ cd ~/tmp +
-  $ ssh login01 wget http://tenet.dl.sourceforge.net/project/scipy/scipy/0.10.1/scipy-0.10.1.tar.gz +
-  $ mv ../scipy-0.10.1.tar.gz ./ +
-  $ tar -xf scipy-0.10.1.tar.gz +
-  $ cd scipy-0.10.1/ +
-  $ python setup.py build --fcompiler=gnu95 #takes about 10 minutes to build +
-  $ python setup.py install --prefix=$HOME/local_dell +
- +
-testing, +
-  $ cd ~/ +
-  $ ipython +
-    import scipy +
-    scipy.__path__ +
-    scipy.test() +
- +
-which did not work 100 % :( +
- +
-FAILED (KNOWNFAIL=13, SKIP=41, failures=2) +
-Out[2]: <nose.result.TextTestResult run=5101 errors=0 failures=1> +
- +
-==== matplotlib / pylab ==== +
- +
-  $ msub -I -l feature=dell +
-  $ cd ~/tmp +
-  $ ssh login01 wget http://ufpr.dl.sourceforge.net/project/matplotlib/matplotlib/matplotlib-1.2.0/matplotlib-1.2.0.tar.gz +
-  $ mv ../matplotlib-1.2.0.tar.gz ./ +
-  $ tar -xf matplotlib-1.2.0.tar.gz +
-  $ cd matplotlib-1.2.0 +
-  $ python setup.py install --prefix=$HOME/local_dell +
- +
-==== mpi4py ==== +
- +
-Is already installed, all that is needed is to load the openmpi module; add the following line to //.bashrc// +
-  module add openmpi-1.6.1-intel +
- +
-==== User Package installation with Anaconda === +
-Modules for anaconda version 2 and 3 are installed on lengau.To load these modules, use the following command (either from the command-line in an interactive session, or in a script: +
-    module add chpc/python/anaconda/2 +
-or: +
-    module add chpc/python/anaconda/3 +
- +
-Using virtualenv with anaconda is not recommended. +
-However, Anaconda does include a similar functionality, which allows users to create an anaconda virtual environment. +
- +
-The following instructions to set up your virtual environment. **YOU NEED ONLY DO THEM ONCE. Please replace USERNAME with your actual username wherever it occurs below.** +
- +
-1. log into lengau +
- +
-2. ''ssh chpclic1'' +
- +
-3. ''module load chpc/python/anaconda/3'' +
- +
-//you can substitute chpc/anaconda/2 if you want python 2// +
- +
-4. ''conda init bash'' +
- +
-5. ''conda create --prefix /home/USERNAME/myenv python=3.6 anaconda'' +
- +
-// change python=3.6 to python=2.7 if you're using anaconda/2 +
- +
-6. ''conda activate /home/USERNAME/myenv'' +
-//You don't have to use "myenv", just replace it consistently with whatever name you prefer for your virtual environment. "myenv" will work, though.// +
- +
-7. You can now install whatever packages you need, that are not installed already for instance, for rpy2, type: +
-    +
-   conda install rpy2 +
- +
-8. you can now type: +
-  conda deactivate /home/USERNAME/myenv +
-to exit the virtual environment. +
- +
- +
-Now, once you've done this, you **must do the following in your job-submit script**: +
-Put it all near the top of your script, but after the #PBS directives: +
- +
-  module add chpc/python/anaconda/3 +
- +
-  conda activate /home/USERNAME/myenv +
- +
-and at the end: +
-  conda deactivate /home/USERNAME/myenv +
- +
-Then submit your script as normal with qsub. +
- +
- +
-if you ever need to install new python packages, just repeat steps 1, 2, 3, 6, 7 and 8 from above. (omit 4 and 5). +
- +
- +
- +
/app/dokuwiki/data/attic/howto/python/start.1566478983.txt.gz · Last modified: 2021/12/09 16:42 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki