wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » guide » python
Trace: • cloud
guide:python

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
guide:python [2020/09/10 13:32]
agill [User Package Installation via pip3 and virtualenv] 		guide:python [2022/06/13 11:58] (current)
wikiadmin
Line 4: Line 4:
 Use this command to get a full list: Use this command to get a full list:
     module avail chpc/python     module avail chpc/python
 +
 +Note that the modules including anaconda are more recent and thus recommended over the plain Python modules, i.e., use one of:
 +
 +  chpc/python/anaconda/3-2019.10
 +  chpc/python/anaconda/3-2020.02
 +  chpc/python/anaconda/3-2020.11
 +  chpc/python/anaconda/3-2021.05
 +  chpc/python/anaconda/3-2021.11
 +
 +>  To check the version of Python provided by each, load the module (after purging any already loaded modules) and then run ''python --version''.  For example (screen capture --- don't type in the ''login1:~$'', that's the shell prompt):
 +
 +  login1:~$ module purge
 +  login1:~$ module load chpc/python/anaconda/3-2019.10
 +  login1:~$ python3 --version
 +  Python 3.7.4
 +  login1:~$ module purge
 +  login1:~$ module load chpc/python/anaconda/3-2021.11
 +  login1:~$ python3 --version
 +  Python 3.9.7
 +  login1:~$ 
 +
 +
  
 All python modules should have mpi4py, numpy and scipy installed; however if matplotlib is required, please use anaconda (which also has the biggest range of 3rd-party python modules). The module ''chpc/python/3.5.2_gcc-6.2.0'' also has a working matplotlib installation. All python modules should have mpi4py, numpy and scipy installed; however if matplotlib is required, please use anaconda (which also has the biggest range of 3rd-party python modules). The module ''chpc/python/3.5.2_gcc-6.2.0'' also has a working matplotlib installation.
Line 24: Line 46:
 You can find more information on user installs in the [[https://packaging.python.org/tutorials/installing-packages/#installing-to-the-user-site|official python documentation]] and more information on pip user installs [[https://pip.pypa.io/en/latest/user_guide/#user-installs|here]]. You can find more information on user installs in the [[https://packaging.python.org/tutorials/installing-packages/#installing-to-the-user-site|official python documentation]] and more information on pip user installs [[https://pip.pypa.io/en/latest/user_guide/#user-installs|here]].
  
-As an alternative, you can also set up a python virtual environment. This essentially gives you your own python installation which you can alter as you see fit. Again, it is perhaps best done in your lustre directory if a large number of python libraries will be installed.+As an alternative, you can also set up a python virtual environment. This essentially gives you your own python installation which you can alter as you see fit. Again, it is perhaps best done in your Lustre directory if a large number of python libraries will be installed. 
 + 
 +To create a virtual environment called //myenv//, just cd to the place where you want the virtual environment to live --- for example 
 + 
 +<code bash> 
 +cd /mnt/lustre/users/USERNAME 
 +</code> 
 + 
 +where ''USERNAME'' is replace by __your__ cluster user name --- and run the following command:
  
-To create a virtual environment called //myenv//, just cd to the place where you want the virtual environment to live (eg. ''cd /mnt/luster/users/USERNAME''), and run the following command: 
  
   python3 -m venv myenv   python3 -m venv myenv
Line 35: Line 64:
  
   source /mnt/lustre/users/USERNAME/myenv/bin/activate   source /mnt/lustre/users/USERNAME/myenv/bin/activate
 +
 +> Note the full path to the virtual environment is on your Lustre directory: ''/mnt/lustre/users/USERNAME/myenv/''
  
 Again, there is more information in [[https://docs.python.org/3/tutorial/venv.html|the official python documentation]]. Again, there is more information in [[https://docs.python.org/3/tutorial/venv.html|the official python documentation]].
Line 72: Line 103:
 6. Activate the environment 6. Activate the environment
     conda activate /home/USERNAME/myenv     conda activate /home/USERNAME/myenv
-You don't have to use "myenv", just replace it consistently with whatever name you prefer for your virtual environment, ''myenv'' will work, though.+You don't have to use "myenv", just replace it consistently with whatever name you prefer for your virtual environment, ''myenv'' will work, though. To enable conda environments, add the following to your script before activating the environment. 
 +    eval "$(conda shell.bash hook)"  
  
 7. You can now install whatever packages you need, that are not installed already; for instance, for rpy2, type: 7. You can now install whatever packages you need, that are not installed already; for instance, for rpy2, type:
/app/dokuwiki/data/attic/guide/python.1599737561.txt.gz · Last modified: 2021/12/09 16:42 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki