======The Lustre Parallel File System======


:!: There is a new Lustre parallel file system mounted on ''/mnt/lustre3p''.  The old Lustre on ''/mnt/lustre'' will be decommissioned after 1 June 2021 and so **//you must copy over the files you will need by// 31 May 2021.** See the [[howto:datamigration|data migration how to]] for more information and guidance on how to copy your data. 



=====Lustre on Lengau=====

Lengau has 16 OSSs arranged in pairs.  There are 28 OSTs connected to the 8 pairs.  Each OST has an approx. capacity of 115 GiB giving a total Lustre storage of 3PiB. 




=====Lustre tools=====

The main tool for displaying information on the files and directories of a Lustre file system is ''lfs''.

====Searching====

Use ''lfs find'' (instead of the standard GNU ''find'' command) to search for Lustre files based on their metadata properties (size, striping, OSTs, etc.).

To find all files larger than 1GB:
  lfs find <dirname> -size +1G

To find all files newer than 60 days:
  lfs find <dirname> -ctime -60


=====Best practices=====

Lustre is designed for parallel access to large files.  It accomplishes this by using //striping// to distribute the contents of a large file across several disk arrays (OSTs) to enable fast parallel IO.

====Striping====

The standard recommendation from the Lustre developers is:

  * Files smaller than 1GB should be left with the default striping (1)
  * Between 1GB and 10GB a small stripe should be used (2)
  * 10 to 100GB : use medium striping (2 to 4)
  * Above 100GB: use maximum striping (-c -1)\\ **Use only this last option with individual files that you know will be above 100GB or use highly parallelized access.**

To set the striping for a file to 2
  lfs setstripe -c 2 <filename>

To change the default striping applied to files in a sub-directory to 2 (and only applied to files larger than 1G)
  lfs setstripe -c 2  -s 1G <dirname>

> Check this ^^^.

**Note:** 

  *When you change the settings for an existing file, the file will actually only change to the new striping when it is recreated.

  * The same applies when you change the settings for an existing directory. You will need to rename the directory, create a new directory with the new stripe settings, and then **copy** (not move) the files to the new directory to inherit the new settings.

See ''lfs migrate'' for more.