wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » survival_guide
Trace:
survival_guide

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
survival_guide [2020/05/26 14:06]
agill [Software Modules on Lengau] 		survival_guide [2022/02/01 18:38] (current)
alopis
Line 8: Line 8:
   - The computers that make up the HPC cluster do not run faster than a good desktop computer, nor do they usually have particularly impressive amounts of memory.  The performance comes from running many compute nodes in parallel.  If your application is not parallelised, it will not run faster at CHPC than on a good desktop machine.   - The computers that make up the HPC cluster do not run faster than a good desktop computer, nor do they usually have particularly impressive amounts of memory.  The performance comes from running many compute nodes in parallel.  If your application is not parallelised, it will not run faster at CHPC than on a good desktop machine.
   - The system is designed to be used in batch mode.  The user prepares a job, submits it to the queue, and the system executes it when the requested amount of resources become available.   - The system is designed to be used in batch mode.  The user prepares a job, submits it to the queue, and the system executes it when the requested amount of resources become available.
-  - You can request an interactive session, which will give you access to a compute node in the cluster.  Here you can compile programs, do pre- or post-processing, and check your job setup.+  - You can request an interactive session, which will give you access to a compute node in the cluster.  Here you can do quick tests or checks, compile programs, do pre- or post-processing, and check your job setup.
   - Access to the system is by means of secure-shell (ssh).   - Access to the system is by means of secure-shell (ssh).
   - Data can be copied to or from the system by means of secure copy (scp) or rsync.   - Data can be copied to or from the system by means of secure copy (scp) or rsync.
Line 19: Line 19:
  
 ===== The user interface (not) ===== ===== The user interface (not) =====
-Although personal computers running Linux have user interfaces that are very similar to Windows or Mac, the ssh-login to the HPC cluster runs in command line text mode only, by default using the Bash shell.  It is therefore essential to master a very small subset of Linux commands.  The internet has many excellent resources explaining how to perform tasks in Linux and Bash.  Nevertheless, here is a very short summary of the most useful instructions: +Although personal computers running Linux have user interfaces that are very similar to Windows or Mac, the ssh-login to the HPC cluster runs in command line text mode only, by default using the Bash shell.  It is therefore essential to master a very small subset of Linux commands.  **PLEASE be aware that there is NO undelete.  If you delete a file from the command line, it is gone, finish, klaar, kaput, never to be seen again.**  The internet has many excellent resources explaining how to perform tasks in Linux and Bash.  Nevertheless, here is a very short summary of the most useful instructions: 
-  * ls - provide a list of the files in the current directory +  * **ls** - provide a list of the files in the current directory 
-  * ls -l - provide a more detailed listing of the files in the current directory +  * **ls -l** - provide a more detailed listing of the files in the current directory 
-  * pwd - gives the current directory path (Present Working Directory) +  * **pwd** - gives the current directory path (Present Working Directory) 
-  * mkdir - create a new directory +  * **mkdir** - create a new directory 
-  * cd - change to a new directory +  * **cd** - change to a new directory 
-  * rmdir - remove a directory +  * **cp** - copy a file (remember to specify the file **and** the location to copy it to) 
-  * rm - remove a file +  * **mv** - move a file (either rename it or cut/paste it into a directory). 
-  * nano - start the nano text editor (recommended for beginners) +  * **rmdir** - remove a directory 
-  * vi - start the vi text editor (warning: steep learning-curve)+  * **rm** - remove a file 
 +  * **nano** - start the nano text editor (recommended for beginners) 
 +  * **vi** - start the vi text editor (warning: steep learning-curve) 
 +  * **qsub** - submit a batch job to PBS for processing 
 +  * **qstat -u $USER** - list the status of your PBS jobs 
 +  * **qdel //pbsjobnumber//** - remove a job from the queue if it is running or terminate it if it is running  
  
 ==== Software Modules on Lengau ==== ==== Software Modules on Lengau ====
Line 34: Line 39:
 Here are examples of the most common module operations that you will need to use on the cluster: Here are examples of the most common module operations that you will need to use on the cluster:
   * **module avail** - To list all available packages   * **module avail** - To list all available packages
 +  * **module avail 2>&1 | grep //searchterm// ** will search through the modules for one containing the expression ''searchterm''.  Use the appropriate name of the software here.
   * **module avail chpc** - List just the chpc modules group - this contains most of the useful modules   * **module avail chpc** - List just the chpc modules group - this contains most of the useful modules
   * **module avail chpc/earth** - This will list all the applications and libraries in the "earth" subgroup (reserved for earth-science applications).   * **module avail chpc/earth** - This will list all the applications and libraries in the "earth" subgroup (reserved for earth-science applications).
Line 39: Line 45:
   * **module load chpc/python/3.5.2_gcc-6.2.0** - An alternative to module add (in this case, loading python version 3.5.2, compiled with gcc-6.2.0)   * **module load chpc/python/3.5.2_gcc-6.2.0** - An alternative to module add (in this case, loading python version 3.5.2, compiled with gcc-6.2.0)
   * **module list** - To list all the modules which you have already loaded in this session   * **module list** - To list all the modules which you have already loaded in this session
-  * **module dell gcc/6.1.0** - To unload a module (in this case gcc version 6.1.0+  * **module unload gcc/6.1.0** - To unload a module (in this case gcc version 6.1.0
   * **module purge** - to unload all modules that you have loaded   * **module purge** - to unload all modules that you have loaded
  
Line 56: Line 62:
 For GCC and the Intel compilers, however, you will have to load the modules that you wish to link your code against separately. For GCC and the Intel compilers, however, you will have to load the modules that you wish to link your code against separately.
  
-Finally, there are a few software packages (mostly those that are commercially licensed, such as Ansys/Fluent, but also a few open-source packages like OpenFOAM) which are installed on Lengau, but not available via the module system. To access these, look at their respective howto pages on this wiki.+Finally, there are a few software packages (mostly those that are commercially licensed, such as Ansys/Fluent, but also a few open-source packages like OpenFOAM) which are installed on Lengau, but not available via the module system. To access these, look at their respective [[howto:|howto]] pages on this wiki.
  
 ==== Installing New Software on Lengau ==== ==== Installing New Software on Lengau ====
Line 66: Line 72:
   * the software available via the debian or ubuntu repositories is not optimised for HPC systems, which have special hardware configurations, and thus special software requirements.   * the software available via the debian or ubuntu repositories is not optimised for HPC systems, which have special hardware configurations, and thus special software requirements.
  
-The CHPC staff will do their best to assist you by installing the software that you request. However this may take considerable time and effort for certain pieces of software to be installed, so you may need to be patient. HPC systems are complex, so unfortunately the installation process is often also complex and tricky. Also, you should ensure that the software which you wish to use will actually benefit from being used on an HPC system. Some software is not written to take advantage of HPC.+The CHPC staff will do their best to assist you by installing the software that you request. However this may take considerable time and effort for certain pieces of software to be installed, so you may need to be patient. HPC systems are complex, so unfortunately the installation process is often also complex and tricky. Also, you should ensure that the software which you wish to use will actually benefit from being used on an HPC system. As mentioned already, some software is not written to take advantage of HPC.
  
 If you wish to use commercially licensed software, you will need to have an appropriate license. Usually this will require a 3-way communication between you, the software licensor and the CHPC. The CHPC does have a small number of national licenses for very popular pieces of software, but is unlikely to procure new licenses of this sort at this time. If you wish to use commercially licensed software, you will need to have an appropriate license. Usually this will require a 3-way communication between you, the software licensor and the CHPC. The CHPC does have a small number of national licenses for very popular pieces of software, but is unlikely to procure new licenses of this sort at this time.
/app/dokuwiki/data/attic/survival_guide.1590494801.txt.gz · Last modified: 2021/12/09 16:42 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki