wiki.chpc.ac.za

This is an old revision of the document!

---

The following CHPC HPC systems (e1350, Blue Gene/P and Sun Microsytems) use schedulers and resources managers to schedule and run jobs in the systems. Below is the detail information:

e1350 cluster - Moab (Scheduler) and Torque (Resource manager).
Blue Gene/P - Loadleveler (Resource manager).
Sun Microsystems - Moab (Scheduler) and Torque (Resource manager).

Herewith an example script of Moab in e1350 cluster:

#!/bin/sh  
#MSUB -l nodes=8:ppn=8
#MSUB -l walltime=100:00:00
#MSUB -m be
#MSUB -V
#MSUB -o /CHPC/work/username/test/test.out
#MSUB -e /CHPC/work/username/test/test.err
#MSUB -d /CHPC/work/username/test
#MSUB -r y
#MSUB -M email.address
#print the time and date
date
cd /CHPC/work/username/test
nproc=`wc $PBS_NODEFILE| awk '{print $1}'`
/CHPC/usr/local/mpiexec-0.83/bin/mpiexec -n $nproc ./EXECUTABLENAME 

Below is an example of Loadleveler script in Blue Gene/P:

#@ job_type = bluegene
#@ bg_size = 64
#@ class = BGP
#@ input = /dev/null
#@ output = dlpoly.$(jobid).out
#@ error = dlpoly.$(jobid).err
#@ wall_clock_limit = 12:00:00
#@ resources = ConsumableCpus(1)
#@ node_usage = shared
#@ cluster_list = BGP
#@ notification = complete
#@ queue
/bgsys/drivers/ppcfloor/bin/mpirun -np 32 -mode VN -cwd /CHPC/work/username/test -exe EXECUTABLENAME 

Below is an example of Moab script in Sun System:

###These lines are for Moab
#MSUB -l nodes=5:ppn=8
#MSUB -l partition=ALL
#MSUB -l walltime=168:00:00
#MSUB -m be
#MSUB -V
#MSUB -o /lustre/SCRATCH2/users/username/work/stdout.out
#MSUB -e /lustre/SCRATCH2/users/username/work/stderr.err
#MSUB -d /lustre/SCRATCH2/users/username/work
#MSUB -mb
#MSUB -M username@mail.co.za
##### Running commands
exe=/opt/gridware/dlpoly/DLPOLY_3.09.Y
nproc=`cat $PBS_NODEFILE | wc -l`
mpirun -np $nproc $exe

There are various controls of Moab and Loadleveler that can help you to manage your job:

Some of Moab controls are as follows:

Commands	Description
msub	        Scheduler job submission
showq	        Show queued jobs
canceljob 	Cancel job
pbsnodes	Check available/unavailable nodes
showres	Show existing reservations
showstart	Show estimate of when job can/will start
showbf	Show current resource availability
Checkjob	Provide report for specified job

Some of Loadleveler controls are as follows:

Commands      Description
llsubmit      submit a job
llq           check status of the job