wiki.chpc.ac.za

Adding Dell Networking N2048 Switch management IP address. This is the BMC network all nodes will be using for IPMI communications.

Connect to the switch using the console and issue the following commands

After plugging nodes on the switch, bright discovers mac address through switch ports

[bright1->device]% newnodes 
MAC First appeared Detected on switch port 
-------------------- -------------------------------- ---------------------------- 
10:98:36:B5:66:93 Fri, 11 Aug 2017 17:51:00 SAST switch01:2
10:98:36:B5:66:A3 Fri, 11 Aug 2017 17:51:29 SAST switch01:3 
10:98:36:B5:66:1F Fri, 11 Aug 2017 17:51:29 SAST switch01:4 
10:98:36:B5:6D:DB Fri, 11 Aug 2017 17:51:59 SAST switch01:5 
10:98:36:B5:6A:FF Fri, 11 Aug 2017 17:51:59 SAST switch01:6 
18:66:DA:94:D4:33 Fri, 11 Aug 2017 17:51:59 SAST switch01:7
18:66:DA:94:D8:D1 Fri, 11 Aug 2017 17:51:00 SAST switch01:8
18:66:DA:94:D3:8B Fri, 11 Aug 2017 17:51:29 SAST switch01:9 
18:66:DA:94:D2:C1 Fri, 11 Aug 2017 17:51:29 SAST switch01:10 
84:7B:EB:F5:CA:46 Fri, 11 Aug 2017 17:51:59 SAST switch01:11
84:7B:EB:F5:D9:02 Fri, 11 Aug 2017 17:51:59 SAST switch01:12

For each node the following is done. Only the procedure for adding “sched” is shown below. “sched” node is connected on switch port 5:

Define “sched”

 [bright1->device]% add physicalnode sched 
 [bright1->device*[sched*]]%

Link the node to the switch port

 [bright1->device*[sched*]]% set ethernetswitch switch01:5 

Configure IPMI Interface

[bright1->device*[sched*]]% interfaces
[bright1->device*[sched*]->interfaces]% add bmc ipmi0
Switched power control for this node to: ipmi0
[bright1->device*[sched*]->interfaces*[ipmi0*]]% set network ipminet
[bright1->device*[sched*]->interfaces*[ipmi0*]]% set ip 172.17.0.5
[bright1->device*[sched*]->interfaces*[ipmi0*]]% exit

Configure Ethernet interface called “bootif”

[bright1->device*[sched*]->interfaces*]% add physical BOOTIF 
[bright1->device*[sched*]->interfaces*[BOOTIF*]]% set network internalnet 
[bright1->device*[sched*]->interfaces*[BOOTIF*]]% set dhcp yes 
[bright1->device*[sched*]->interfaces*[BOOTIF*]]% exit

Configure Infinabnd Interface

[bright1->device*[sched*]->interfaces*]% add physical ib0 
[bright1->device*[sched*]->interfaces*[ib0*]]% set network ibnet 
[bright1->device*[sched*]->interfaces*[ib0*]]% set dhcp yes 
[bright1->device*[sched*]->interfaces*[ib0*]]% exit

List defined Network interfaces

[bright1->device[sched]->interfaces]% list 
Type         Network device name  IP               Network 
------------ -------------------- ---------------- ---------------- 
bmc          ipmi0                172.17.0.5       ipminet 
physical     BOOTIF [prov,dhcp]   172.18.0.221     internalnet 
physical     ib0 [dhcp]           0.0.0.0          ibnet 
[bright1->device[sched]->interfaces]%

NB: ib network ip is issued when OFED is installed. Click Infiniband Configuration to view the steps

Set provisioning Interface

 [bright1->device*[sched*]]% set provisioninginterface bootif 

Set node category. (node categories are linked to software images)

 [bright1->device*[sched*]]% set category scheduler 

Commit all the chages

 [bright1->device*[sched*]]% commit 

The node begins to install after the “commit” command