wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » quick » pbspro
Trace: • compiling_r • pbspro • data • compiling_octave • induction • feedback • compiling_gcc • new • moab • mfa
quick:pbspro

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
quick:pbspro [2015/05/06 12:09]
ischeepers [How to submit a batch job] 		quick:pbspro [2025/09/22 17:47] (current)
kevin [Example 4 : empty job]
Line 1: Line 1:
 ====== Job submission using PBSPro ====== ====== Job submission using PBSPro ======
  
-As of January 2014, CHPC has a new scheduler - PBSPro.+The CHPC system uses the PBSPro scheduler.
  
-Main user documentation can be found at http://resources.altair.com/pbs/documentation/support/PBSProUserGuide12.1.pdf+Main user documentation can be found at [[https://www.altair.com/pbs-works-documentation/|Altair]]
  
 But here is a summary of how to submit jobs at CHPC. But here is a summary of how to submit jobs at CHPC.
Line 26: Line 26:
 A script can include just about anything which you could do during a terminal session such as set environment variables, change directories, and move files.  A script can include just about anything which you could do during a terminal session such as set environment variables, change directories, and move files. 
  
-''qsub myscriptname.sub''+> Job script files can be named to anything allowed for a regular file, but it is strongly recommended that you use ''.pbs'' as the file extension to make it easy to distinguish job scripts from other shell scripts (''.sh'').
  
  
Line 50: Line 50:
  
 ''qstat -xf <jobnum>'' ''qstat -xf <jobnum>''
 +
 +====How to monitor job arrays====
 +
 +To see the array job status on the queue, postfix the
 +jobnumber with empty square brackets:
 +
 +''qstat -J <jobnum>[]''
 +
 +
 +To look at one specific job in the array, add ''-t'' to the qstat arguments:
 +
 +''qstat -xft <jobnum>[array-index]''
 +
 +
 +eg
 +
 +''qstat -xft 967890[2]''
 +
  
  
Line 58: Line 76:
  
 ====Job Exit Codes >= 128 (or 256)==== ====Job Exit Codes >= 128 (or 256)====
-This means the job was killed with a signal. The signal is given by X modulo 128 (or 256). For example an exit value of 137 means the job's top process was killed with signal 9 (137 mod 128 = 9). +This means the job was killed by the PBS scheduler by sending it a signal.  
-The exit status values greater than 128 (or 256) indicate which signal killed the job. Depend- ing on the system, values greater than 128 (or on some systems 256; see man wait(2) for more information), are the value of the signal that killed the job. (i.e., 265 mod 256 = 9 or 271 mod 256 = 15.)+ 
 +The **modulo** or **remainder** operator [[http://en.wikipedia.org/wiki/Modulo_operation]] is used to interpret (or “decode”) PBS job exit values.  
 + 
 + 
 +The signal is given by X modulo 128 (or 256). An exit value of 137 means the job's top process was killed with signal 9 because ''137 mod 128 = 9''.  
 + 
 +Depending on the system, a value greater than 128 or 256, see ''man wait(2)'', is the signal that was sent by PBS-Pro to kill the job. (i.e., 265 mod 256 = 9 or 271 mod 256 = 15.) 
  
-To interpret (or “decode”) the signal contained in the exit status value, subtract the base value from the exit status. For example, if a job had an exit status of 143, that indicates the job was killed via a SIGTERM (e.g. 143 - 128 = 15, signal 15 is SIGTERM). See the kill(1) manual page for a mapping of signal numbers to signal name on your operating system.+Exit code 271 means that the job exceeded the limit requested at submission, eg. walltime or cpu usage, and was killed.
  
 +See ''man kill'' for a mapping of signal numbers to signal name on your operating system.
 ====Terminate a job by signal==== ====Terminate a job by signal====
 Send a signal to the job Send a signal to the job
Line 84: Line 109:
  
  
-===== Example 2 (a) :  Amber job on GPU nodes (NB: For other executables e.g. python and sender, see example 2(b)) =====+===== Examples: Amber jobs =====
  
-<code> 
-#!/bin/sh 
-#PBS -N eric.job 
-#PBS -l select=1:ncpus=12:mpiprocs=12 
-#PBS -l walltime=1:00:00 
-#PBS -q kepla_k20 
-#PBS -m be 
-#PBS -o /export/home/username/amber/amber12/eric.out 
-#PBS -e /export/home/username/amber/amber12/eric.err 
-#PBS -M email@mail.com 
  
-cd $PBS_O_WORKDIR+[[howto:amber|Amber job submission scripts]]
  
-pwd 
-echo "My job starts here" 
-date 
-source /etc/profile.d/modules.sh 
-module add intel-XE/c8000 cuda/5.0-c8000 mvapich2/c8000 amber/k20 
-module add amber/k20 
  
-exe=pmemd.cuda.MPI 
-nproc=`cat $PBS_NODEFILE|wc -l` 
-time mpirun -np $nproc -machinefile $exe -O -i mdin -o mdout -p prmtop -c inpcrd 
-echo "My job ends here" 
-date 
-</code> 
  
-===== Example 2 (b) :  Amber job on normal nodes (Please use this for python,sander and other executables that do not work on GPU===== +===== Example: Quantum Espresso =====
- +
-<code> +
-#!/bin/sh +
-#PBS -N eric.job +
-#PBS -l select=1:ncpus=12:mpiprocs=12 +
-#PBS -l walltime=1:00:00 +
-#PBS -q workq +
-#PBS -m abe +
-#PBS -o /export/home/username/amber/amber12/eric.out +
-#PBS -e /export/home/username/amber/amber12/eric.err +
-#PBS -M email@mail.com+
  
-cd $PBS_O_WORKDIR 
  
-pwd+[[howto:quantum_espresso|Quantum Espresso]]
  
-source /etc/profile.d/modules.sh 
-module add intel-XE/13.0 openmpi/openmpi-1.6.5-intel amber/12 
  
-echo "My job starts here" +===== Example 4 : empty job =====
-date +
- +
-exe=sander.MPI +
- +
-nproc=`cat $PBS_NODEFILE|wc -l` +
-time mpirun -np $nproc -machinefile $exe -O -i mdin -o mdout -p prmtop -c inpcrd +
-echo "My job ends here" +
-date +
-</code> +
- +
-===== Example 3 : Quantum Espresso job ===== +
- +
-<code> +
-#!/bin/sh +
-###These lines are for PBSpro +
-#PBS -N QEspresso +
-#PBS -l select=2:ncpus=12:mpiprocs=12:jobtype=dell,place=free:group=nodetype:excl +
-#PBS -l walltime=24:00:00 +
-#PBS -q workq +
-#PBS -m be +
-#PBS -o /export/home/username/scratch/run-directory/stdout +
-#PBS -e /export/home/username/scratch/run-directory/stderr +
-#PBS -V +
-##### Environment Settings +
-source /etc/profile.d/modules.sh +
-module add intel-XE/13.0 +
-module add QEspresso/5.0.2 +
-##### Running commands +
-NP=`cat $PBS_NODEFILE | wc -l` +
-cd $HOME/scratch5 +
-mpirun -f $PBS_NODEFILE -r ssh -ib -env I_MPI_DEBUG 2 -np $NP pw.x -npool 3  -inp inputfile.inp  +
-</code> +
- +
-===== Example 4 :  empty job =====+
  
 <code> <code>
Line 186: Line 141:
 #<hosts> = number of hosts #<hosts> = number of hosts
 #<architecture> = architecture where the jobs will run (nehalem, westmere, dell) #<architecture> = architecture where the jobs will run (nehalem, westmere, dell)
-#<queue> = submission queue ( workq , priorityq , specialq , intel_mic , spark , kepla_k20 , accelrys)+#<queue> = submission queue ( serial, smp, normal, ...,  accelrys)
 #<executable> = Binary to be run #<executable> = Binary to be run
  
Line 193: Line 148:
 </code> </code>
  
-//Note that the M9000 sparc queue is called 'spark'// 
  
-===== MPI example =====+ 
 +===== Example 5 : MPI example =====
  
 Syntax Syntax
Line 205: Line 160:
 ... ...
 </code> </code>
- 
 ===== Further examples ===== ===== Further examples =====
   * Various [[howto:bioinformatics?&#basic_examples|bioinformatics examples]] illustrate some further possibilities. Specifically the blast examples illustrate job arrays and job dependencies.   * Various [[howto:bioinformatics?&#basic_examples|bioinformatics examples]] illustrate some further possibilities. Specifically the blast examples illustrate job arrays and job dependencies.
   * This [[howto:gpu_gromacs#pbs_job_script|gromacs example]] shows an interesting case where mpi, openmp and the gpus are used in a single job.   * This [[howto:gpu_gromacs#pbs_job_script|gromacs example]] shows an interesting case where mpi, openmp and the gpus are used in a single job.
/app/dokuwiki/data/attic/quick/pbspro.1430906949.txt.gz · Last modified: 2021/12/09 16:42 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki