This is an old revision of the document!
Table of Contents
The Lustre Parallel File System
Lustre on Lengau
Lustre tools
The main tool for information on the files and directories of a Lustre file system is lfs.
Searching
Use lfs find (instead of the standard GNU find command) to search for Lustre files based on their metadata properties (size, striping, OSTs, etc.).
To find all files larger than 1GB:
lfs find <dirname> -size +1G
To find all files newer than 60 days:
lfs find <dirname> -ctime -60
Best practices
Lustre is designed for parallel access to large files. It accomplishes this by using striping to distribute the contents of a large file across several disk arrays (OSTs) to enable fast parallel IO.
Striping
The standard recommendation from the Lustre developers is:
- Files smaller than 1GB should be left with the default striping (1)
- Between 1GB and 10GB a small stripe should be used (2)
- 10 to 100GB : use medium striping (2 to 4)
- Above 100GB: use maximum striping (-c -1)
Use only this last option with individual files that you know will be above 100GB or use highly parallelized access.
To set the striping for a file to 2
lfs setstripe -c 2 <filename>
To change the default striping applied to files in a sub-directory to 2 (and only applied to files larger than 1G)
lfs setstripe -c 2 -s 1g <dirname>
