User Tools

Site Tools


acelab:nfs_configuration

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
acelab:nfs_configuration [2018/07/27 19:14]
smasoka created
acelab:nfs_configuration [2018/07/27 19:52] (current)
smasoka
Line 1: Line 1:
-On the Host servers [nfs01 and nfs02]+**On the Host servers [nfs01 and nfs02]**
  
 Device Mapper Multipathing Devices (DMMP) Device Mapper Multipathing Devices (DMMP)
  
-Scan for virtual disks + 
-Display the multipath device topology +Scan for virtual disks \\ 
-Create a partition on a multipath device node +Display the multipath device topology ​\\ 
-Add a partition to Device Mapper(DM) +Create a partition on a multipath device node \\ 
-Crate a file systems on a Device Mapper partition +Add a partition to Device Mapper(DM) ​\\ 
-Mount a Device mapper partition+Crate a file systems on a Device Mapper partition ​\\ 
 +Mount a Device mapper partition ​\\
  
 1. Install all packages that are needed 1. Install all packages that are needed
 +<​code>​
 [root@nfs02 ~]# yum install device-mapper device-mapper-devel device-mapper-multipath kpartx [root@nfs02 ~]# yum install device-mapper device-mapper-devel device-mapper-multipath kpartx
 [root@nfs02 ~]# yum install sg3_utils sg3_utils-devel ​ [root@nfs02 ~]# yum install sg3_utils sg3_utils-devel ​
 +</​code>​
  
 2. Start multipath service 2. Start multipath service
 +<​code>​
 [root@nfs02 ~]# service multipathd start [root@nfs02 ~]# service multipathd start
 Starting multipathd daemon: ​                               [  OK  ] Starting multipathd daemon: ​                               [  OK  ]
 [root@nfs02 ~]# chkconfig multipathd on [root@nfs02 ~]# chkconfig multipathd on
 [root@nfs02 ~]# [root@nfs02 ~]#
 +</​code>​
  
 3. Scan for devices and partitions 3. Scan for devices and partitions
 +<​code>​
 [root@nfs02 ~]# rescan-scsi-bus.sh ​ [root@nfs02 ~]# rescan-scsi-bus.sh ​
 Scanning SCSI subsystem for new devices Scanning SCSI subsystem for new devices
Line 93: Line 99:
 0 device(s) removed. ​                 0 device(s) removed. ​                
 [root@nfs02 ~]# [root@nfs02 ~]#
 +</​code>​
  
  
 4. Display device mappings 4. Display device mappings
 +<​code>​
 [root@nfs02 ~]# multipath -ll [root@nfs02 ~]# multipath -ll
 mpathe (3600a098000b6278f00000240599ffeea) dm-3 DELL,MD34xx mpathe (3600a098000b6278f00000240599ffeea) dm-3 DELL,MD34xx
Line 122: Line 129:
 `-+- policy='​round-robin 0' prio=9 status=enabled `-+- policy='​round-robin 0' prio=9 status=enabled
   `- 11:0:1:1 sdg 8:96  active ready running   `- 11:0:1:1 sdg 8:96  active ready running
 +</​code>​
  
 5. Grab the image of nfs02and reboot nfs01 just to check if all is well.  5. Grab the image of nfs02and reboot nfs01 just to check if all is well. 
 +<​code>​
 [root@nfs01 ~]# multipath -ll [root@nfs01 ~]# multipath -ll
 mpathe (3600a098000b6278f00000240599ffeea) dm-3 DELL,MD34xx mpathe (3600a098000b6278f00000240599ffeea) dm-3 DELL,MD34xx
Line 151: Line 159:
   `- 11:0:1:1 sdg 8:96  active ready running   `- 11:0:1:1 sdg 8:96  active ready running
 [root@nfs01 ~]# [root@nfs01 ~]#
 +</​code>​
  
 6. Storage Array Virtual Disks - Using Dell Storage Manager 6. Storage Array Virtual Disks - Using Dell Storage Manager
 +<​code>​
 Disk /​dev/​mapper/​mpathc:​ 1.5 TB APPS Disk /​dev/​mapper/​mpathc:​ 1.5 TB APPS
 Disk /​dev/​mapper/​mpathd:​ 200 GB PBS_SHARED Disk /​dev/​mapper/​mpathd:​ 200 GB PBS_SHARED
 Disk /​dev/​mapper/​mpathe:​ 680 GB CM_SHARED Disk /​dev/​mapper/​mpathe:​ 680 GB CM_SHARED
 Disk /​dev/​mapper/​mpathf:​ 2.5 TB HOME Disk /​dev/​mapper/​mpathf:​ 2.5 TB HOME
 +</​code>​
 +
  
 7. Install "​parted"​ if not already installed 7. Install "​parted"​ if not already installed
 +<​code>​
 [root@nfs01 ~]# yum install parted parted-devel [root@nfs01 ~]# yum install parted parted-devel
 +</​code>​
  
 8. Create partitions 8. Create partitions
 +<​code>​
 [root@nfs01 ~]# parted /​dev/​mapper/​mpathc ​ [root@nfs01 ~]# parted /​dev/​mapper/​mpathc ​
 GNU Parted 2.1 GNU Parted 2.1
Line 181: Line 196:
  
 [root@nfs01 ~]# [root@nfs01 ~]#
 +</​code>​
 +
  
 9. Create a XFS File system from the partition 9. Create a XFS File system from the partition
 +<​code>​
 [root@nfs01 ~]# mkfs.xfs -L APPS /​dev/​mapper/​mpathcp1 ​ [root@nfs01 ~]# mkfs.xfs -L APPS /​dev/​mapper/​mpathcp1 ​
 meta-data=/​dev/​mapper/​mpathcp1 ​  ​isize=256 ​ agcount=4, agsize=100663168 blks meta-data=/​dev/​mapper/​mpathcp1 ​  ​isize=256 ​ agcount=4, agsize=100663168 blks
Line 193: Line 211:
 realtime =none                 ​extsz=4096 ​  ​blocks=0,​ rtextents=0 realtime =none                 ​extsz=4096 ​  ​blocks=0,​ rtextents=0
 [root@nfs01 ~]# [root@nfs01 ~]#
 +</​code>​
  
 10. Crate partitions and file system for all virtual disks [PBS_SHARED,​CM_SHARED,​HOME] 10. Crate partitions and file system for all virtual disks [PBS_SHARED,​CM_SHARED,​HOME]
 +
 11. Mount on /etc/fstab using LABELS - this is just for testing 11. Mount on /etc/fstab using LABELS - this is just for testing
 +<​code>​
 [root@nfs01 ~]# cat /etc/fstab |grep LABEL [root@nfs01 ~]# cat /etc/fstab |grep LABEL
 LABEL=APPS ​                   /apps                           ​xfs ​       defaults ​   0 0 LABEL=APPS ​                   /apps                           ​xfs ​       defaults ​   0 0
Line 217: Line 238:
 /​dev/​mapper/​mpathfp1 ​ 2.5T   ​33M ​ 2.5T   1% /nfshome /​dev/​mapper/​mpathfp1 ​ 2.5T   ​33M ​ 2.5T   1% /nfshome
 [root@nfs01 ~]# [root@nfs01 ~]#
 +</​code>​
  
 12. Add mount points to fsmounts on device nfs01 using bright 12. Add mount points to fsmounts on device nfs01 using bright
 +<​code>​
 [bright1->​device[nfs01]->​fsmounts]% add /apps        [bright1->​device[nfs01]->​fsmounts]% add /apps       
 [bright1->​device*[nfs01*]->​fsmounts*[/​apps*]]% set device /​dev/​mapper/​mpathcp1 ​                     ​ [bright1->​device*[nfs01*]->​fsmounts*[/​apps*]]% set device /​dev/​mapper/​mpathcp1 ​                     ​
Line 253: Line 276:
 /​dev/​mapper/​mpathfp1 ​             /​nfshome ​                        ​xfs ​             /​dev/​mapper/​mpathfp1 ​             /​nfshome ​                        ​xfs ​            
 [bright1->​device[nfs01]->​fsmounts]% [bright1->​device[nfs01]->​fsmounts]%
 +</​code>​
  
 13. The above step does the following (as soon as you commit 13. The above step does the following (as soon as you commit
 +<​code>​
 [root@nfs01 ~]# df -h|grep path [root@nfs01 ~]# df -h|grep path
 /​dev/​mapper/​mpathcp1 ​ 1.5T   ​33M ​ 1.5T   1% /apps /​dev/​mapper/​mpathcp1 ​ 1.5T   ​33M ​ 1.5T   1% /apps
Line 266: Line 291:
 /​dev/​mapper/​mpathfp1 ​            /​nfshome ​            ​xfs ​         defaults ​        0 0 /​dev/​mapper/​mpathfp1 ​            /​nfshome ​            ​xfs ​         defaults ​        0 0
 [root@nfs01 ~]# [root@nfs01 ~]#
 +</​code>​
  
 14. Set up nfsexports through bright 14. Set up nfsexports through bright
 +<​code>​
 [bright1->​device*[nfs01*]->​fsexports*[/​apps*]]% set hosts 172.20.0.0/​24 [bright1->​device*[nfs01*]->​fsexports*[/​apps*]]% set hosts 172.20.0.0/​24
 [bright1->​device*[nfs01*]->​fsexports*[/​apps*]]% set async yes [bright1->​device*[nfs01*]->​fsexports*[/​apps*]]% set async yes
Line 299: Line 326:
  
 [root@nfs01 ~]# [root@nfs01 ~]#
 +</​code>​
  
 15. Mount the fsmounts point [on all node categories] 15. Mount the fsmounts point [on all node categories]
 +<​code>​
 [bright1->​category]% foreach compute iml login scheduler (fsmounts; add /home; set device nfs01.ib.cluster:/​nfshome;​ set filesystem nfs; commit) ​ [bright1->​category]% foreach compute iml login scheduler (fsmounts; add /home; set device nfs01.ib.cluster:/​nfshome;​ set filesystem nfs; commit) ​
 [bright1->​category]% foreach compute iml login scheduler (fsmounts; add /cm/shared; set device nfs01.ib.cluster:/​cmshared;​ set filesystem nfs; commit) [bright1->​category]% foreach compute iml login scheduler (fsmounts; add /cm/shared; set device nfs01.ib.cluster:/​cmshared;​ set filesystem nfs; commit)
Line 308: Line 335:
 [bright1->​category]% foreach scheduler (fsmounts; add /pbsshared; set device nfs01.ib.cluster:/​pbsshared;​ set filesystem nfs; commit) [bright1->​category]% foreach scheduler (fsmounts; add /pbsshared; set device nfs01.ib.cluster:/​pbsshared;​ set filesystem nfs; commit)
 [bright1->​category]% ​ [bright1->​category]% ​
 +</​code>​
/var/www/wiki/data/attic/acelab/nfs_configuration.1532711696.txt.gz · Last modified: 2018/07/27 19:14 by smasoka