The Lustre Parallel File System

There is a new Lustre parallel file system mounted on /mnt/lustre3p. The old Lustre on /mnt/lustre will be decommissioned after 1 June 2021 and so you must copy over the files you will need by 31 May 2021. See the data migration how to for more information and guidance on how to copy your data.

Lengau has 16 OSSs arranged in pairs. There are 28 OSTs connected to the 8 pairs. Each OST has an approx. capacity of 115 GiB giving a total Lustre storage of 3PiB.

The main tool for displaying information on the files and directories of a Lustre file system is lfs.

Use lfs find (instead of the standard GNU find command) to search for Lustre files based on their metadata properties (size, striping, OSTs, etc.).

To find all files larger than 1GB:

lfs find <dirname> -size +1G

To find all files newer than 60 days:

lfs find <dirname> -ctime -60

Lustre is designed for parallel access to large files. It accomplishes this by using striping to distribute the contents of a large file across several disk arrays (OSTs) to enable fast parallel IO.

The standard recommendation from the Lustre developers is:

• Files smaller than 1GB should be left with the default striping (1)
• Between 1GB and 10GB a small stripe should be used (2)
• 10 to 100GB : use medium striping (2 to 4)
• Above 100GB: use maximum striping (-c -1)
  Use only this last option with individual files that you know will be above 100GB or use highly parallelized access.

To set the striping for a file to 2

lfs setstripe -c 2 <filename>

To change the default striping applied to files in a sub-directory to 2 (and only applied to files larger than 1G)

lfs setstripe -c 2  -s 1G <dirname>

Check this ^^^.

Note:

• When you change the settings for an existing file, the file will actually only change to the new striping when it is recreated.

• The same applies when you change the settings for an existing directory. You will need to rename the directory, create a new directory with the new stripe settings, and then copy (not move) the files to the new directory to inherit the new settings.

See lfs migrate for more.