wiki.chpc.ac.za

To execute MPI programs you will need to submit a job script to the scheduler. There is an example PBSPro job script on the school00 home directory called example.sh and which can be copied to your current directory in the usual way:

cp ~/../student00/example.sh .

example.sh:

#!/bin/bash
#PBS -N example01
#PBS -l select=1:ncpus=24:mpiprocs=24
#PBS -P Wchpc
#PBS -q R522145
#PBS -W group_list=training
#PBS -l walltime=4:00:00
 
CWD=/mnt/lustre/users/$USER/example
 
 
 
cd $CWD
echo "Working directory is $CWD"
nproc=`cat $PBS_NODEFILE | wc -l`
echo "nproc is $nproc"
echo "Nodes used by this job:"
cat $PBS_NODEFILE
 
echo "Starting run..."
# Insert command to run here:
 
echo "... done."

Note that the options passed to the qsub command appear as shell comment lines that start with #PBS.

Note: the script uses example/ as the working directory. Create this before you run the script:

mkdir ~/lustre/example

or edit the script file to point to the prac directory.

NOTE: your script will fail if you don't use the Lustre file system.

In the example file the command to actually run your code is missing. Depending on which part of the prac you are attempting, this will differ. Insert the command in the indicated place.

To then submit the job script to the scheduler you simply use

qsub example.sh

The qsub command will read all its option parameters from the file.

To check the status of your jobs in the Winter School queue:

qstat | grep R522145

This command lists all the jobs in the R522145 queue. Running jobs will have a R in the status column, a Q if the job is queued and waiting.

Use the qdel command to remove a queue job from the queue. The synatx is

qdel jobid

where jobid is replaced by the number of the job (looks like 557809.sched01).