User Tools

Site Tools


acelab:adding_new_nodes

All nodes are configured with BMC ip address. Below is the table with all IP address (Including the compute nodes of the Power Edge M1000e)

Add main switch to Bright Cluster Manager

Adding Dell Networking N2048 Switch management IP address. This is the BMC network all nodes will be using for IPMI communications.

Connect to the switch using the console and issue the following commands

Mangement Nodes

Node Names BMC IP's
bright1 172.17.0.1
bright2 172.17.0.2
login01 172.17.0.3
login02 172.17.0.4
sched 172.17.0.5
imlnode 172.17.0.6
nfs01 172.17.0.7
nfs02 172.17.0.8
mds01 172.17.0.9
mds02 172.17.0.10
oss01 172.17.0.11
oss02 172.17.0.12

After plugging nodes on the switch, bright discovers mac address through switch ports

[bright1->device]% newnodes 
MAC First appeared Detected on switch port 
-------------------- -------------------------------- ---------------------------- 
10:98:36:B5:66:93 Fri, 11 Aug 2017 17:51:00 SAST switch01:2
10:98:36:B5:66:A3 Fri, 11 Aug 2017 17:51:29 SAST switch01:3 
10:98:36:B5:66:1F Fri, 11 Aug 2017 17:51:29 SAST switch01:4 
10:98:36:B5:6D:DB Fri, 11 Aug 2017 17:51:59 SAST switch01:5 
10:98:36:B5:6A:FF Fri, 11 Aug 2017 17:51:59 SAST switch01:6 
18:66:DA:94:D4:33 Fri, 11 Aug 2017 17:51:59 SAST switch01:7
18:66:DA:94:D8:D1 Fri, 11 Aug 2017 17:51:00 SAST switch01:8
18:66:DA:94:D3:8B Fri, 11 Aug 2017 17:51:29 SAST switch01:9 
18:66:DA:94:D2:C1 Fri, 11 Aug 2017 17:51:29 SAST switch01:10 
84:7B:EB:F5:CA:46 Fri, 11 Aug 2017 17:51:59 SAST switch01:11
84:7B:EB:F5:D9:02 Fri, 11 Aug 2017 17:51:59 SAST switch01:12

For each node the following is done. Only the procedure for adding “sched” is shown below. “sched” node is connected on switch port 5:

Define “sched”

 [bright1->device]% add physicalnode sched 
 [bright1->device*[sched*]]%

Link the node to the switch port

 [bright1->device*[sched*]]% set ethernetswitch switch01:5 

Define network interfaces for the node

Configure IPMI Interface

[bright1->device*[sched*]]% interfaces
[bright1->device*[sched*]->interfaces]% add bmc ipmi0
Switched power control for this node to: ipmi0
[bright1->device*[sched*]->interfaces*[ipmi0*]]% set network ipminet
[bright1->device*[sched*]->interfaces*[ipmi0*]]% set ip 172.17.0.5
[bright1->device*[sched*]->interfaces*[ipmi0*]]% exit

Configure Ethernet interface called “bootif”

[bright1->device*[sched*]->interfaces*]% add physical BOOTIF 
[bright1->device*[sched*]->interfaces*[BOOTIF*]]% set network internalnet 
[bright1->device*[sched*]->interfaces*[BOOTIF*]]% set dhcp yes 
[bright1->device*[sched*]->interfaces*[BOOTIF*]]% exit

Configure Infinabnd Interface

[bright1->device*[sched*]->interfaces*]% add physical ib0 
[bright1->device*[sched*]->interfaces*[ib0*]]% set network ibnet 
[bright1->device*[sched*]->interfaces*[ib0*]]% set dhcp yes 
[bright1->device*[sched*]->interfaces*[ib0*]]% exit

List defined Network interfaces

[bright1->device[sched]->interfaces]% list 
Type         Network device name  IP               Network 
------------ -------------------- ---------------- ---------------- 
bmc          ipmi0                172.17.0.5       ipminet 
physical     BOOTIF [prov,dhcp]   172.18.0.221     internalnet 
physical     ib0 [dhcp]           0.0.0.0          ibnet 
[bright1->device[sched]->interfaces]%

NB: ib network ip is issued when OFED is installed. Click Infiniband Install and Configure to view the steps

Set provisioning Interface

 [bright1->device*[sched*]]% set provisioninginterface bootif 

Set node category. (node categories are linked to software images)

 [bright1->device*[sched*]]% set category scheduler 

Commit all the chages

 [bright1->device*[sched*]]% commit 

The node begins to install after the “commit” command

Compute Node

Node Names BMC IP's
cnode01 172.17.0.15
cnode02 172.17.0.16
cnode03 172.17.0.17
cnode04 172.17.0.18
cnode05 172.17.0.19
cnode06 172.17.0.20
cnode07 172.17.0.21
cnode08 172.17.0.22
cnode09 172.17.0.23
cnode10 172.17.0.24
cnode11 172.17.0.25
cnode12 172.17.0.26
cnode13 172.17.0.27
cnode14 172.17.0.28
cnode15 172.17.0.29
cnode16 172.17.0.30
/var/www/wiki/data/pages/acelab/adding_new_nodes.txt · Last modified: 2018/07/27 18:56 by smasoka