wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » quick » pbspro
Trace: • m9000 • connect • loadshedding • cfdchallenge
quick:pbspro

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
quick:pbspro [2015/06/11 09:34]
ischeepers [Job Exit Codes >= 128 (or 256)] 		quick:pbspro [2025/09/22 17:47] (current)
kevin [Example 4 : empty job]
Line 1: Line 1:
 ====== Job submission using PBSPro ====== ====== Job submission using PBSPro ======
  
-As of January 2014, CHPC has a new scheduler - PBSPro.+The CHPC system uses the PBSPro scheduler.
  
-Main user documentation can be found at http://resources.altair.com/pbs/documentation/support/PBSProUserGuide12.1.pdf+Main user documentation can be found at [[https://www.altair.com/pbs-works-documentation/|Altair]]
  
 But here is a summary of how to submit jobs at CHPC. But here is a summary of how to submit jobs at CHPC.
Line 26: Line 26:
 A script can include just about anything which you could do during a terminal session such as set environment variables, change directories, and move files.  A script can include just about anything which you could do during a terminal session such as set environment variables, change directories, and move files. 
  
-''qsub myscriptname.sub''+> Job script files can be named to anything allowed for a regular file, but it is strongly recommended that you use ''.pbs'' as the file extension to make it easy to distinguish job scripts from other shell scripts (''.sh'').
  
  
Line 50: Line 50:
  
 ''qstat -xf <jobnum>'' ''qstat -xf <jobnum>''
 +
 +====How to monitor job arrays====
 +
 +To see the array job status on the queue, postfix the
 +jobnumber with empty square brackets:
 +
 +''qstat -J <jobnum>[]''
 +
 +
 +To look at one specific job in the array, add ''-t'' to the qstat arguments:
 +
 +''qstat -xft <jobnum>[array-index]''
 +
 +
 +eg
 +
 +''qstat -xft 967890[2]''
 +
  
  
Line 91: Line 109:
  
  
-===== Example 2 (a) :  Amber job on GPU nodes (NB: For other executables e.g. python and sender, see example 2(b)) =====+===== Examples: Amber jobs =====
  
-<code> 
-#!/bin/sh 
-#PBS -N eric.job 
-#PBS -l select=1:ncpus=12:mpiprocs=12 
-#PBS -l walltime=1:00:00 
-#PBS -q kepla_k20 
-#PBS -m be 
-#PBS -o /export/home/username/amber/amber12/eric.out 
-#PBS -e /export/home/username/amber/amber12/eric.err 
-#PBS -M email@mail.com 
  
-cd $PBS_O_WORKDIR+[[howto:amber|Amber job submission scripts]]
  
-pwd 
-echo "My job starts here" 
-date 
-source /etc/profile.d/modules.sh 
-module add intel-XE/c8000 cuda/5.0-c8000 mvapich2/c8000 amber/k20 
-module add amber/k20 
  
-exe=pmemd.cuda.MPI 
-nproc=`cat $PBS_NODEFILE|wc -l` 
-time mpirun -np $nproc -machinefile $exe -O -i mdin -o mdout -p prmtop -c inpcrd 
-echo "My job ends here" 
-date 
-</code> 
  
-===== Example 2 (b) :  Amber job on normal nodes (Please use this for python,sander and other executables that do not work on GPU=====+===== Example: Quantum Espresso =====
  
-<code> 
-#!/bin/sh 
-#PBS -N eric.job 
-#PBS -l select=1:ncpus=12:mpiprocs=12 
-#PBS -l walltime=1:00:00 
-#PBS -q workq 
-#PBS -m abe 
-#PBS -o /export/home/username/amber/amber12/eric.out 
-#PBS -e /export/home/username/amber/amber12/eric.err 
-#PBS -M email@mail.com 
  
-cd $PBS_O_WORKDIR+[[howto:quantum_espresso|Quantum Espresso]]
  
-pwd 
- 
-source /etc/profile.d/modules.sh 
-module add intel-XE/13.0 openmpi/openmpi-1.6.5-intel amber/12 
- 
-echo "My job starts here" 
-date 
- 
-exe=sander.MPI 
- 
-nproc=`cat $PBS_NODEFILE|wc -l` 
-time mpirun -np $nproc -machinefile $exe -O -i mdin -o mdout -p prmtop -c inpcrd 
-echo "My job ends here" 
-date 
-</code> 
- 
-===== Example 3 : Quantum Espresso job ===== 
- 
-<code> 
-#!/bin/sh 
-###These lines are for PBSpro 
-#PBS -N QEspresso 
-#PBS -l select=2:ncpus=8:mpiprocs=8:jobtype=nehalem 
-#PBS -l walltime=24:00:00 
-#PBS -q workq 
-#PBS -m be 
-#PBS -o stdout 
-#PBS -e stderr 
-##### Environment Settings 
-source /etc/profile.d/modules.sh 
-module add intel-XE/13.0 
-module add QEspresso/5.0.2 
-##### Running commands 
-NP=`cat $PBS_NODEFILE | wc -l` 
-cd $PBS_O_WORKDIR 
-mpirun -f $PBS_NODEFILE -r ssh -ib -env I_MPI_DEBUG 2 -np $NP pw.x -npool 3  -inp inputfile.inp  
-</code> 
  
 ===== Example 4 : empty job ===== ===== Example 4 : empty job =====
Line 192: Line 141:
 #<hosts> = number of hosts #<hosts> = number of hosts
 #<architecture> = architecture where the jobs will run (nehalem, westmere, dell) #<architecture> = architecture where the jobs will run (nehalem, westmere, dell)
-#<queue> = submission queue ( workq , priorityq , specialq , intel_mic , spark , kepla_k20 , accelrys)+#<queue> = submission queue ( serial, smp, normal, ...,  accelrys)
 #<executable> = Binary to be run #<executable> = Binary to be run
  
Line 199: Line 148:
 </code> </code>
  
-//Note that the M9000 sparc queue is called 'spark'//+
  
 ===== Example 5 : MPI example ===== ===== Example 5 : MPI example =====
Line 211: Line 160:
 ... ...
 </code> </code>
- 
-===== Example 6 : job array ===== 
-See [[howto:pbs-pro_job_submission_examples#job_arrays|here]] for a longer explanation. 
-<file bash simple_job_array.pbs> 
-#! /bin/bash 
-#PBS -l select=1:ncpus=1 
-#PBS -l walltime=00:01:00 
-#PBS -o /export/home/username/scratch5/simple_jobarray.stdout 
-#PBS -e /export/home/username/scratch5/simple_jobarray.stderr 
-#PBS -M youremail@address.com 
-#PBS -m abe 
-#PBS -N J_example 
-#PBS -J 1-24 
- 
- 
-# Make sure we're in the right working directory 
-cd /export/home/username/scratch5/ 
- 
-#sleep a random number of seconds (<10) 
-sleep $[ ( $RANDOM % 10 )  + 1 ]s 
- 
-#see what happens when we write to a file... 
-echo "Wawaweewaa! This is sub-job ${PBS_ARRAY_INDEX} of job ${PBS_JOBID} running on ${HOSTNAME} run by ${USER} writing to stdout!" 
-echo "Wawaweewaa! This is sub-job ${PBS_ARRAY_INDEX} of job ${PBS_JOBID} running on ${HOSTNAME} run by ${USER} writing to stderr!" 1>&2 
-echo "Wawaweewaa! This is sub-job ${PBS_ARRAY_INDEX} of job ${PBS_JOBID} running on ${HOSTNAME} run by ${USER} appending to simple_jobarray.txt" >> simple_jobarray.txt 
-</file> 
- 
-Thing to note: the stdout and stderr outputs do not all get joined together as they might with an mpi job. 
- 
 ===== Further examples ===== ===== Further examples =====
   * Various [[howto:bioinformatics?&#basic_examples|bioinformatics examples]] illustrate some further possibilities. Specifically the blast examples illustrate job arrays and job dependencies.   * Various [[howto:bioinformatics?&#basic_examples|bioinformatics examples]] illustrate some further possibilities. Specifically the blast examples illustrate job arrays and job dependencies.
   * This [[howto:gpu_gromacs#pbs_job_script|gromacs example]] shows an interesting case where mpi, openmp and the gpus are used in a single job.   * This [[howto:gpu_gromacs#pbs_job_script|gromacs example]] shows an interesting case where mpi, openmp and the gpus are used in a single job.
/app/dokuwiki/data/attic/quick/pbspro.1434008066.txt.gz · Last modified: 2021/12/09 16:42 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki