Managing data is an integral part of doing computational analysis work. Scientific applications generally use and create very large amounts of raw data. The graphs and tables that eventually show up in reports, theses and research publications represent this data in post-processed form. However, the raw data generated in the process of may take up several Terabytes of data. Once the raw data has been post-processed, it is seldom necessary to keep it, and it should be removed as soon as possible.

On the Lengau cluster, users have access to a total of 4 Petabytes of “scratch” storage space. The hardware underpinning this storage is a Lustre storage cluster, consisting of meta-data servers, object storage servers (OSSs), object storage targets (OSTs), a high-speed Infiniband network and several thousand spinning hard drives. To the user it looks like a single file system. This 4 PB scratch space is intended purely for short term storage. The system prioritises speed over reliability. The CHPC's mostly unread policy document states clearly that files older than 90 days may be removed at the CHPC's discretion.