Differences

This shows you the differences between two versions of the page.

--- guide:lustre [2020/04/29 14:53]
kevin created
+++ guide:lustre [2021/12/09 16:42] (current)
@@ Line 1: / Line 1: @@
 ======The Lustre Parallel File System======
+:!: There is a new Lustre parallel file system mounted on ''/mnt/lustre3p''.  The old Lustre on ''/mnt/lustre'' will be decommissioned after 1 June 2021 and so **//you must copy over the files you will need by// 31 May 2021.** See the [[howto:datamigration|data migration how to]] for more information and guidance on how to copy your data.
+=====Lustre on Lengau=====
+Lengau has 16 OSSs arranged in pairs.  There are 28 OSTs connected to the 8 pairs.  Each OST has an approx. capacity of 115 GiB giving a total Lustre storage of 3PiB.
+=====Lustre tools=====
+The main tool for displaying information on the files and directories of a Lustre file system is ''lfs''.
+====Searching====
+Use ''lfs find'' (instead of the standard GNU ''find'' command) to search for Lustre files based on their metadata properties (size, striping, OSTs, etc.).
+To find all files larger than 1GB:
+  lfs find <dirname> -size +1G
+To find all files newer than 60 days:
+  lfs find <dirname> -ctime -60
+=====Best practices=====
+Lustre is designed for parallel access to large files.  It accomplishes this by using //striping// to distribute the contents of a large file across several disk arrays (OSTs) to enable fast parallel IO.
+====Striping====
+The standard recommendation from the Lustre developers is:
+  * Files smaller than 1GB should be left with the default striping (1)
+  * Between 1GB and 10GB a small stripe should be used (2)
+  * 10 to 100GB : use medium striping (2 to 4)
+  * Above 100GB: use maximum striping (-c -1)\\ **Use only this last option with individual files that you know will be above 100GB or use highly parallelized access.**
+To set the striping for a file to 2
+  lfs setstripe -c 2 <filename>
+To change the default striping applied to files in a sub-directory to 2 (and only applied to files larger than 1G)
+  lfs setstripe -c 2  -s 1G <dirname>
+> Check this ^^^.
+**Note:**
+  *When you change the settings for an existing file, the file will actually only change to the new striping when it is recreated.
+  * The same applies when you change the settings for an existing directory. You will need to rename the directory, create a new directory with the new stripe settings, and then **copy** (not move) the files to the new directory to inherit the new settings.
+See ''lfs migrate'' for more.

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools