User Tools

Site Tools


gpu:file_systems

The GPU cluster is provisioned with a set of two shared filesystems mounted on the head node as well as on all the GPU compute nodes as follows :

Filesystem            Size  Used Avail Use% Mounted on 
/dev/gpfs              12T   45G   12T   1% /GPU  
sn8-ib@o2ib:/lustre   7.9T  5.7G  7.5T   1% /LUSTRE 

The “/GPU” filesystem of 12TB is served by a set of four (4) GPFS NSD Servers in a GPFS storage cluster. This mount is shared between all the nodes (hn, cn1, cn2,cn3,cn4,cn5). System administrators should refer to the seperate Wiki namespace on GPFS for configuration, management and operations of the GPFS cluster.

The “/LUSTRE” filesystem mount is a 7.9TB shared filesystem served by a Lustre cluster running on a single shared Lustre MDS/MGS, plus a set of 3 x ODS servers, serving a total of 9 Lustre OST's (3 per ODS). System administrators can refer to the Wiki namespace on Lustre for configuration, management and operations of the Lustre cluster.

Connectivity between GPFS clients, Lustre clients and the respective storage cluster servers uses 40Gbps QDR Infiniband, with GPFS using Direct Memory Access (RDMA) over Infiniband.

/var/www/wiki/data/pages/gpu/file_systems.txt · Last modified: 2010/10/08 17:39 by mlawrence