User Tools

Site Tools


guide:filesystems

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
guide:filesystems [2016/05/25 11:20]
wikiadmin
guide:filesystems [2020/05/18 16:10] (current)
kevin
Line 8: Line 8:
 | "​Apps" ​ | ''/​apps'' ​ | NFS  | 20 TB  | none  | Yes  | On request ​ |  | "​Apps" ​ | ''/​apps'' ​ | NFS  | 20 TB  | none  | Yes  | On request ​ | 
 | "​Data" ​ | ''/​lustre/​data'' ​ | Lustre ​ | 1 PB  | none  | NO  | On request only  |  | "​Data" ​ | ''/​lustre/​data'' ​ | Lustre ​ | 1 PB  | none  | NO  | On request only  | 
-| "​BSP" ​ | ''/​lustre/​bsp'' ​ | Lustre ​ | 226 TB  | none  | NO  | BSP only  | +
  
 In the above table **Name** refers to the colloquial name of the file system, i.e., what it is referred to in ordinary speech or writing. In the above table **Name** refers to the colloquial name of the file system, i.e., what it is referred to in ordinary speech or writing.
  
 +=====Home=====
 +
 +The home file system contains all users' home directories as is usual for a Linux system. ​ This is where your standard and primary files reside, most especially your [[guide:​connect|ssh]] keys file, your login scripts and other configuration files.
 +
 +====Quota====
 +
 +Your home directory is subject to a strict quota. ​ Once you exceed that quota you will not be able to write any more to any file within that directory. ​ This has implications ​ for files that log various things, e.g. your command line history, and installed applications.
 +
 +For this reason, large application codes should be installed in your Lustre directory, or you should use a pre-installed application from the Apps file system.
 +
 +To request that an application be installed in Apps please open a [[https://​users.chpc.ac.za/​helpdesk/​tickets/​submit/​|Helpdesk]] ticket.
 +
 + 
 +
 +=====Lustre=====
 +
 +The Lustre file system is a high performance parallel file system which is to be used for all running jobs on the Lengau cluster.
 +
 +This is a work space, or "​scratch"​ file system and is **not** intended for long term storage. ​ Any files you store on Lustre must be required for your jobs.  Files that have not been //​accessed//​ for more than 60 days will be considered stale and may be deleted if the Lustre file systems fills up (exceeds 80% usage).
 +
 +Note that Lustre has a complex structure and is best suited for large files and parallel access. See the separate [[guide:​lustre|Lustre guide]] for more information and important tips on use. 
 +
 +====Job Scripts====
 +
 +The //working directory// for all your job scripts must be on Lustre. ​ Each user is allocated a directory on the Lustre file system under
 +
 +  /​mnt/​lustre/​users/​username
 +
 +where ''​username''​ is replaced by //your// user name.
 +
 +Large codes, data sets, and output data must be stored under your Lustre directory. ​ This is why we often refer to Lustre as the "​work"​ file system---it'​s where all your work resides.
 +
 +>  Smaller applications,​ scripts, libraries, etc. can be stored in your home directory, of course.
 +
 +====Lustre is NOT Backed Up====
 +
 +We do not keep a back up of Lustre. ​ Any loss of files and data is entirely **unrecoverable**. ​ It is thus essential that you take data management seriously and ensure that you have adequate off site storage to maintain your own copies of any important files or data on Lustre.
 +
 +Lustre is designed for //​performance//​ and capacity.
 +
 +To upload or download data to/from Lustre use the ''​scp''​ login node for small files (~10GB) or the Globus system for **large** files (>​100GB). ​ See the [[guide:​connect|Connection Guide]] for more details and important tips.
 +
 +
 +=====Apps=====
 +
 +The Apps file system contains application codes, libraries, development tools, compilers, etc., installed by the CHPC for our users.
 +
 +This system is accessed through the GNU modules tool ''​module''​. ​ Examples are provided in the [[quick:​start|Quick Start Guide]].
 +
 +=====Data=====
  
 +By special arrangement research groups may store files and data that are shared by members of that group for longer term (up to 365 days).
  
/var/www/wiki/data/attic/guide/filesystems.1464168007.txt.gz · Last modified: 2016/05/25 11:20 by wikiadmin