wiki.chpc.ac.za

This is an old revision of the document!

---

The following CHPC HPC systems (e1350, Blue Gene/P and Sun Microsytems) use schedulers and resources managers to schedule and run jobs in the systems. Below is the detail information:

e1350 cluster - Moab (Scheduler) and Torque (Resource manager).
Blue Gene/P - Loadleveler (Resource manager).
Sun Microsystems - Moab (Scheduler) and Torque (Resource manager).

Herewith an example script of Moab in e1350 cluster:

#!/bin/sh  
#MSUB -l nodes=8:ppn=8
#MSUB -l walltime=100:00:00
#MSUB -m be
#MSUB -V
#MSUB -o /CHPC/work/username/test/test.out
#MSUB -e /CHPC/work/username/test/test.err
#MSUB -d /CHPC/work/username/test
#MSUB -r y
#MSUB -M email.address
#print the time and date
date
cd /CHPC/work/username/test
nproc=`wc $PBS_NODEFILE| awk '{print $1}'`
/CHPC/usr/local/mpiexec-0.83/bin/mpiexec -n $nproc ./EXECUTABLENAME 

Below is an example of Loadleveler script in Blue Gene/P:

#@ job_type = bluegene
#@ bg_size = 64
#@ class = BGP
#@ input = /dev/null
#@ output = dlpoly.$(jobid).out
#@ error = dlpoly.$(jobid).err
#@ wall_clock_limit = 12:00:00
#@ resources = ConsumableCpus(1)
#@ node_usage = shared
#@ cluster_list = BGP
#@ notification = complete
#@ queue
/bgsys/drivers/ppcfloor/bin/mpirun -np 32 -mode VN -cwd /CHPC/work/username/test -exe EXECUTABLENAME 

Below is an example of Moab script in Sun System:

###These lines are for Moab
#MSUB -l nodes=5:ppn=8
#MSUB -l partition=ALL
#MSUB -l walltime=168:00:00
#MSUB -m be
#MSUB -V
#MSUB -o /lustre/SCRATCH2/users/username/work/stdout.out
#MSUB -e /lustre/SCRATCH2/users/username/work/stderr.err
#MSUB -d /lustre/SCRATCH2/users/username/work
#MSUB -mb
#MSUB -M username@mail.co.za
##### Running commands
exe=/opt/gridware/dlpoly/DLPOLY_3.09.Y
nproc=`cat $PBS_NODEFILE | wc -l`
mpirun -np $nproc $exe

There are various controls of Moab and Loadleveler that can help you to manage your job:

Some of Moab controls are as follows:

Commands	  Description
msub	          Scheduler job submission
showq	          Show queued jobs
canceljob 	  Cancel job
pbsnodes	  Check available or unavailable nodes
showres	  Show existing reservations
showstart	  Show estimate of when job can or will start
showbf	  Show current resource availability
Checkjob	  Provide report for specified job

Some of Loadleveler controls are as follows:

Commands      Description
llsubmit      submit a job
llq           check status of the job