Differences

This shows you the differences between two versions of the page.

--- howto:pbsdsh [2023/10/10 12:48]
wikiadmin created
+++ howto:pbsdsh [2026/08/25 21:58] (current)
kevin [Example 2: job script]
@@ Line 4: / Line 4: @@
 Users should use the ''pbsdsh'' command to launch multiple serial processes, especially across many nodes.
+=====Syntax=====
+''pbsdsh'' [ ''-n'' //<vnode index>// ] [''-s''] [''-v''] [''-o''] ''-''''-'' //<program>// [//<program args>//]
+=====With GNU parallel=====
 However, ''pbsdsh'' is very simple so it's helpful to use ''parallel'' as the task scheduler and ''pbsdsh'' to launch the executable on each node.
-Example:
+====Example 1====
+Say you have a shell script ''script.sh'' that you want to run repeatedly with different arguments (parameters).  Store those different parameter sets, one per line, in a plain text file ''inputs.txt''.
+Then you can use GNU ''parallel'' to use ''pbsdsh'' repeatedly to launch the script with a different line of parameters across 24 cores (i.e., one node) with
 <code bash>
-SCRIPT=$PWD/script.sh # Script to run.
+parallel -j 24 pbsdsh -n {%} -- bash -l -c '"'script.sh {}'"' :::: inputs.txt
-INPUTS=inputs.txt # Each line in this file is used as arguments to ${SCRIPT}
+</code>
-# It's fine to have more input lines than you have requested cpus,
-# extra jobs will be executed as cpus become available
-# Here '{%}' gets replaced with the job slot ({1..$PBS_NCPUS})
+Note that this works even if you have more input lines than the available CPUs (24 in one regular compute node), because ''parallel -j 24'' will only execute 24 at a time, and the extra jobs will be executed as cores become available.
-# and '{}' gets replaced with a line from ${INPUTS}.
-#
-# Pbsdsh starts a very minimal shell. `bash -l` loads all of your startup files, so that things like modules work.
-# The `-c` is so that bash separates out the arguments correctly (otherwise they're all in a single string)
-parallel -j ${PBS_NCPUS} pbsdsh -n {%} -- bash -l -c '"'${SCRIPT} {}'"' :::: ${INPUTS}
+GNU parallel will replace the ''{%}'' with the job slot number, from 1 to maximum cores (24 in the example).
+and ''{}'' gets replaced with a line from ''inputs.txt''.
+Pbsdsh starts a very minimal shell, so we evoke ''bash -l'' to start up a full bash shell which loads all of your startup files, so that things like modules work.
+The ''-c'' option to ''bash'' is so that bash separates out the arguments correctly (otherwise they're all in a single string).
+====Example 2: job script====
+<code bash>
+#!/bin/bash
+#PBS -N p2nodes
+#PBS -l select=2:ncpus=24:mpiprocs=24
+#PBS -P YOURPROJECT
+#PBS -q normal
+#PBS -l walltime=0:30:00
+CWD=/mnt/lustre/users/$USER/example
+cd $CWD
+echo "Working directory: " `pwd`
+nprocs=`cat $PBS_NODEFILE | wc -l`
+SCDIR=/mnt/lustre/users/$USER/scripts
+SCRIPT=$SCDIR/script.sh     # Script to run.
+INPUTS=inputs.txt           # Each line in this file is used as arguments to ${SCRIPT}
+parallel -j ${nprocs} pbsdsh -n {%} -- bash -l -c '"'${SCRIPT} {}'"' :::: ${INPUTS}
 </code>
-Ref.: [[https://opus.nci.org.au/display/Help/GNU+parallel|GNU parallel]] at NCI.
+===Ref.===
+[[https://opus.nci.org.au/display/Help/GNU+parallel|GNU parallel]] at NCI.

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools