User Tools

Site Tools


howto:tipsandtricks

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
howto:tipsandtricks [2020/04/01 16:11]
ccrosby [Checking for zombie processes]
howto:tipsandtricks [2021/07/19 12:47] (current)
ccrosby
Line 1: Line 1:
 ====== Tips and Tricks ====== ====== Tips and Tricks ======
 +
 +===== Managing your storage usage =====
 +You may want to do yourself the favour of checking with the commercial cloud providers how much they charge for data storage. ​ Once you have picked yourself up off the floor, set about managing your level of usage of the CHPC's **free** Lustre storage resource. ​
 +
 +Remember that the CHPC Lustre is **not** intended for long-term storage - it is temporary workspace, is limited and has been designed to be fast rather than reliable. ​ The CHPC's official policies allow us to remove data that has not been used in the preceding 90 days.  Please be pro-active about managing your data before we do it for you.  **Without** first asking your permission. ​ In order to get a list of files that have not been accessed in the last 90 days, use the ''​find''​ command:  ​
 +<​code>​
 +find  -type f -atime +90 -printf "​%h/​%f,​ %s, %AD \n"
 +</​code> ​
 +which will produce a csv table with the path, size in bytes and last time of access. ​ To make it even easier for yourself, simply delete these files automagically with find:
 +<​code>​
 +find  -type f -atime +90 -delete
 +</​code>​
 +
 +
 +
 +===== Navigating around your files and directories =====
 +If you are unfamiliar with the Linux command line, it can be painful to find your way around your files and directories. ​ Consider using the [[http://​linuxcommand.org/​lc3_adv_mc.php|Gnu Midnight Commander]]. ​ It is available on Lengau by way of a module: ​
 +<​code>​
 +module load chpc/​mc/​4.8.17
 +</​code>​
 +
 +
 +===== Do not put echo statements in your .bashrc file =====
 +It is tempting to put echo statements in your .bashrc file, so that you can get a handy heads-up when you log in that some environment variables are being set.  Please do not do this.  The reason is that it breaks scp, which  expects to see its protocol data over the stdin/​stdout channels. ​ There are ways around this (you can use Google, right?), but unless you know what you are doing, just don't put echo statements in a .bashrc file.
  
  
Line 104: Line 128:
   - Install [[https://​github.com/​billziss-gh/​winfsp/​releases/​tag/​v1.2POST1|WinFSP]]   - Install [[https://​github.com/​billziss-gh/​winfsp/​releases/​tag/​v1.2POST1|WinFSP]]
   - Install [[https://​github.com/​billziss-gh/​sshfs-win/​releases/​latest|SSHFS-Win]]   - Install [[https://​github.com/​billziss-gh/​sshfs-win/​releases/​latest|SSHFS-Win]]
-  - Use your Windows file explorer'​s "Home - Easey Acess - Map as Drive" menu sequence to get a panel where you can input your scp.chpc.ac.za login credentials and select a drive letter. ​ You will need to tick "​Connect using different credentials"​. +  - Use your Windows file explorer'​s "Home - Easy Acess - Map as Drive" menu sequence to get a panel where you can input your scp.chpc.ac.za login credentials and select a drive letter. ​ You will need to tick "​Connect using different credentials"​. 
-  - The above process will mount your home directory. ​ If you want your lustre directory instead, append ..\..\mnt\lustre\users to the path.+  - The above process will mount your home directory. ​ If you want your lustre directory instead, append ..\..\mnt\lustre3p\users to the path.
   - **Please** don't use the login ID "​joeblogs"​ ...   - **Please** don't use the login ID "​joeblogs"​ ...
  
Line 117: Line 141:
 You **may** have difficulties submitting a PBS job from a compute node, although as the cluster is currently configured, it generally does work.  However, it is possible to have an ssh command in a PBS script, so the obvious solution if you experience difficulties,​ is to ssh to the login node in order to submit another PBS script, if you wanted to submit another PBS script at the completion of the current one, you could insert a line like this at the end of your first script: You **may** have difficulties submitting a PBS job from a compute node, although as the cluster is currently configured, it generally does work.  However, it is possible to have an ssh command in a PBS script, so the obvious solution if you experience difficulties,​ is to ssh to the login node in order to submit another PBS script, if you wanted to submit another PBS script at the completion of the current one, you could insert a line like this at the end of your first script:
 <​code>​ <​code>​
-ssh login2 qsub /mnt/lustre/​users/​jblogs/​scripts/​PBS_Script1+ssh login2 qsub /mnt/lustre3p/​users/​jblogs/​scripts/​PBS_Script1
 </​code>​ </​code>​
  
/var/www/wiki/data/attic/howto/tipsandtricks.1585750299.txt.gz · Last modified: 2020/04/01 16:11 by ccrosby