User Tools

Site Tools


howto:data

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
howto:data [2020/06/26 11:05]
ccrosby
howto:data [2020/09/09 12:30] (current)
ccrosby [The CHPC's limited storage capacity]
Line 24: Line 24:
 ===== The CHPC's limited storage capacity ===== ===== The CHPC's limited storage capacity =====
 On the Lengau cluster, users have access to a total of 4 Petabytes of "​scratch"​ storage space. ​ The hardware underpinning this storage is a Lustre storage cluster, consisting of meta-data servers, object storage servers (OSSs), object storage targets (OSTs), a high-speed Infiniband network and several thousand spinning hard drives. ​ To the user it looks like a single file system. ​ This 4 PB scratch space is intended purely for short term storage. ​ The system prioritises speed over reliability. ​ The CHPC's mostly unread policy document states clearly that files older than 90 days may be removed at the CHPC's discretion. On the Lengau cluster, users have access to a total of 4 Petabytes of "​scratch"​ storage space. ​ The hardware underpinning this storage is a Lustre storage cluster, consisting of meta-data servers, object storage servers (OSSs), object storage targets (OSTs), a high-speed Infiniband network and several thousand spinning hard drives. ​ To the user it looks like a single file system. ​ This 4 PB scratch space is intended purely for short term storage. ​ The system prioritises speed over reliability. ​ The CHPC's mostly unread policy document states clearly that files older than 90 days may be removed at the CHPC's discretion.
 +
 +Please be pro-active about managing your data before we do it for you.  **Without** first asking your permission. ​ In order to get a list of files that have not been accessed in the last 90 days, use the ''​find''​ command:  ​
 +<​code>​
 +find  -type f -atime +90 -printf "​%h/​%f,​ %s, %AD \n"
 +</​code> ​
 +which will produce a csv table with the path, size in bytes and last time of access. ​ To make it even easier for yourself, simply delete these files automagically with find:
 +<​code>​
 +find  -type f -atime +90 -delete
 +</​code>​
 +
 +
 +
  
 =====Compressing Files===== =====Compressing Files=====
/var/www/wiki/data/attic/howto/data.1593162301.txt.gz · Last modified: 2020/06/26 11:05 by ccrosby