This is an old revision of the document!
scratch5Gaussian can be run over single nodes or more than one node (see examples below), with an example input file below these. These two case are also possible using Job Arrays (see example scripts further below) which allow multiple jobs to be submitted as if they were one job (containing sub-jobs) and which are much easier for the PBSPro scheduler to handle than the same number of individual jobs. Hence you will be able to submit many jobs and not be constrained by the 10 job limit for each user - and this won't be a problem for the system either!
There are two modules of Gaussian 09 at CHPC which can be used. Here is an example on how to access them:
embele@login01:~/scratch5 $ module avail ### list available modules embele@login01:~/scratch5 $ module add gaussian/g09.D01 ### load g09 version D01 (previous) or embele@login01:~/scratch5 $ module add gaussian/g09.anton ### load g09 version anton (new, Recommended)
<file bash gaussian_single.qsub>
#! /bin/sh
# (1) These are standard type lines for PBS job specification. Modify them for your own requirements - choice of
# nodes; walltime; your directory on scratch for standard out and error files ; email of job starting,
# ending and errors
#PBS -N molecule.job
#PBS -l select=1:ncpus=12:jobtype=westmere,place=free:group=nodetype
#PBS -l walltime=2:31:00
#PBS -q workq
#PBS -o /export/home/aslopis/scratch5/Gaussian_Support/C60_4/molecule.out
#PBS -e /export/home/aslopis/scratch5/Gaussian_Support/C60_4/molecule.err
#PBS -m abe
#PBS -M MY_EMAIL_ADDRESS.com
# End of comments (1) Make changes above.
# (2) This is to clean up temporary files which are left behind if Gaussian crashes. No changes required below.
cleanup_scratch() {
echo "Deleting inside signal handler, meaning I either hit the walltime, or deleted the job using qdel"
cd "$PBS_O_WORKDIR"
rm -rfv "$GAUSS_SCRDIR"
echo "---"
echo "Signal handler ending time:"
date
exit 0
}
# Associate function "cleanup_scratch" with the TERM signal (and others), which is usually how jobs get killed.
trap 'cleanup_scratch' SIGTERM SIGHUP SIGINT SIGQUIT SIGFPE SIGKILL SIGALRM
# End of comments (2) No changes required above.
# (3) You can remove this 1st line below after you've run the script once, but is no problem if it remains.
mkdir /lustre/SCRATCH5/gau/$USER
# (4) Please leave this line unchanged. Creating a subdirectory with your jobid for Gaussian temporary files
# which are removed after job.
# Previously everone's Gaussian temporary files were written in /SCRATCH5/gau
mkdir /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# (5) Change to your work directory. Specify if explictly if you find issues.
cd $PBS_O_WORKDIR
echo "My job starts here"
date
# (6) Please leave unchanged. This new module specifies the new directory for Gaussian variable GAUSS_SCRDIR
# for temporary files directory.
source /etc/profile.d/modules.sh
module add gaussian/g09.anton
source /opt/gridware/applications/gaussian/g09/bsd/g09.profile
# (6) Please leave unchanged
# (7) Edit your job input and log file names here.
g09 < molecule.com > molecule.log
# (8) Leave following unchanged.
# Normal exit of script: clean up temporary files - Gaussian should do this anyway, but to be safe.
# Remove the Gaussian scratch directory where temporary files are written. It is
# /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# If abnormal termination (walltime or job killed) then function subroutine cleanup_scratch will clear
# temporary files.
rm -rfv "$GAUSS_SCRDIR"
echo "Normal execution ending time:"
date
Please include the following line at the top of your input file (an example .com is provided below):
%lindaworkers=LINDA
<file bash gaussian_multiple.qsub>
#!/bin/bash
# (1) These are standard type lines for PBS job specification. Modify them for your own requirements - choice of
# nodes; walltime; your directory on scratch for standard out and error files ; email of job starting,
# ending and errors.
#PBS -N molecule.job
#PBS -l select=2:ncpus=12:jobtype=westmere,place=free:group=nodetype
#PBS -l walltime=03:59:00
#PBS -q workq
#PBS -m abe
#PBS -o /export/home/aslopis/scratch5/Gaussian_Support/C60_1/molecule.out
#PBS -e /export/home/aslopis/scratch5/Gaussian_Support/C60_1/molecule.err
#PBS -M MY_Email_Address.com
# End of comments (1) Make changes above.
# (2) This is to clean up temporary files which are left behind if Gaussian crashes. No changes required below.
cleanup_scratch() {
echo "Deleting inside signal handler, meaning probably either hit walltime, or deleted the job using qdel"
cd "$PBS_O_WORKDIR"
rm -rfv "$GAUSS_SCRDIR"
echo "---"
echo "Signal handler ending time:"
date
exit 0
}
# Associate function "cleanup_scratch" with the TERM signal (and others), which is usually how jobs get killed.
trap 'cleanup_scratch' SIGTERM SIGHUP SIGINT SIGQUIT SIGFPE SIGKILL SIGALRM
# End of comments (2) No changes required above.
# (3) You can remove this first line below after you've run the script once, but won't be problem if it remains.
mkdir /lustre/SCRATCH5/gau/$USER
# (4) Please leave this line unchanged. Creating a subdirectory with your jobid for temporary Gaussian files
# which are removed after job.
# Previously everone's Gaussian temporary files were written in /SCRATCH5/gau
mkdir /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# (5) Change to your work directory. Specify if explictly if you find issues.
# cd "$PBS_O_WORKDIR"
cd /export/home/aslopis/scratch5/Gaussian_Support/C60_1
echo "My job starts here"
date
pwd
# (6) Please leave unchanged. This new module specifies the new directory for Gaussian variable GAUSS_SCRDIR
# for temporary files directory.
source /etc/profile.d/modules.sh
module add gaussian/g09.anton
source /opt/gridware/applications/gaussian/g09/bsd/g09.profile
# (6) Please leave unchanged
# (7) Edit your job input and log file names here. In this these are molecule.com and molecule.log.
# Please leave remaining text unchanged.
LINDA=`cat $PBS_NODEFILE | uniq | tr '\n' "," | sed 's|,$||' `
echo linda: $LINDA
cat molecule.com | sed "s/LINDA/$LINDA/" > temp$$.inp
g09 < temp$$.inp > molecule.log
# (7) Edit as specified above please.
# (8) Leave following unchanged.
# Normal exit of script: clean up temporary files - Gaussian should do this anyway, but to be safe.
# Remove the Gaussian scratch directory where temporary files are written. It is
# /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# If abnormal termination (walltime or job killed) then function subroutine cleanup_scratch will clear
# temporary files.
rm -rfv "$GAUSS_SCRDIR"
echo "Normal execution ending time:"
date
%nprocshared=12
%nprocl=1
#P HF/6-31G* IOP(6/33=2,6/41=10,6/42=17) SCF=Tight Pop=MK
Title Card Required
0 1
C -3.19550100 0.11344600 -0.18511100
O -3.05859100 0.83554400 -1.15941100
N -2.14200500 -0.25446800 0.60848900
H -2.32841200 -0.78506500 1.44398900
C -0.75900100 0.12341400 0.36048900
C 0.14379200 -0.42259800 1.48678900
C 0.59047400 -1.84630300 1.03238900
C 1.59689900 0.10938400 1.28248900
C 0.08697100 -2.05959700 -0.42431100
C 2.05278100 -1.31402200 0.82788900
C -0.14011000 -0.59319400 -0.87441100
C 1.35866600 -2.48501300 -1.17731100
C 1.32989700 -0.05621300 -1.07501100
C 1.66721100 0.99508300 0.02128900
C 2.20428100 -1.28752400 -0.71551100
C 0.60162500 2.08159700 0.02358900
N -0.63668200 1.58111300 0.28218900
O 0.86964000 3.24909300 -0.21751100
H -1.43757500 2.10622300 -0.04521100
O 2.93691800 1.56466600 -0.15981100
H 2.76333000 2.51016900 -0.30861100
H -0.28050600 -0.28619200 2.48278900
H 0.43696300 -2.68340100 1.71298900
H 2.07960500 0.56667700 2.14378900
H -0.78403700 -2.70728500 -0.52611100
H 2.86017500 -1.79913300 1.37408900
H -0.75990800 -0.47138600 -1.76101100
H 1.22276500 -2.52601100 -2.26181100
H 1.75375300 -3.44771800 -0.83891100
H 1.50280200 0.34398500 -2.07391100
H 3.22968200 -1.22473700 -1.07621100
C -4.55170800 -0.42883700 0.22928900
H -5.23089700 0.41157200 0.38338900
H -4.52881600 -1.04383700 1.13088900
H -4.94951500 -1.02253200 -0.59631100
#! /bin/sh
#PBS -N My.job
#PBS -l select=1:ncpus=12:jobtype=westmere,place=free:group=nodetype:excl
#PBS -l walltime=0:10:00
#PBS -q workq
#PBS -m abe
#PBS -M MyEmailAddress
# Any of these comment lines can remain in place and won't interfere with the script - or removed if you prefer.
# The line below defined the job array. In this example there are 3 sub-jobs. Each sub-job is essentially
# an individual job but which uses the same settings such as node, walltime and email about jobs.
# "qstat -ftu username" is an appropriate way to check on your job array status, each subjob will be listed
# as [1], [2] etc. You can do a qdel on a sub-job, just use job number including square brackets.
#PBS -J 1-3
# This is to clean up temporary files which are left behind if Gaussian crashes. No changes required below.
cleanup_scratch() {
echo "Deleting inside signal handler, meaning I probably either hit the walltime, or deleted the job using qdel"
cd "$PBS_O_WORKDIR"
rm -rfv "$GAUSS_SCRDIR"
echo "---"
echo "Signal handler ending time:"
date
exit 0
}
# Associate the function "cleanup_scratch" with the TERM signal, which is usually how jobs get killed
trap 'cleanup_scratch' TERM
source /etc/profile.d/modules.sh
module add gaussian/g09.anton
source /opt/gridware/applications/gaussian/g09/bsd/g09.profile
mkdir /lustre/SCRATCH5/gau/$USER
mkdir /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# Change to your work directory (usually more reliable than using "$PBS_O_WORKDIR"). Write your exact work
# directory in the format below using /lustre
cd /lustre/SCRATCH5/users/aslopis/Job_Array_Testing
# These are arbitrary subdirectory names for each sub-job, please set up all your input files (.com and
# .chk) in this subdirectory and your output files will be written here. Hence each sub-job will be in
# separate directories and more clearly organised. The names are arbitrary, so you should
# name them in a meaningful way for you. Please note, the input files must also have the exact same name as
# the directory containing them so the script can then find and run using them. In the current example only
# 3 jobs are specified (PBS -J 1-3), but you can specify as many directories/sub-jobs as you like (more than
# 23 if desired). This bypasses the 10 queued job limit (counts as 1 job) but is not problematic for the
# system - in fact better!
directory[1]=alpha
directory[2]=beta
directory[3]=gamma
directory[4]=delta
directory[5]=epsilon
directory[6]=eta
directory[7]=theta
directory[8]=iota
directory[9]=kappa
directory[10]=lamda
directory[11]=mu
directory[12]=nu
directory[13]=xi
directory[14]=omicron
directory[15]=pi
directory[16]=rho
directory[17]=sigma
directory[18]=tau
directory[19]=upsilon
directory[20]=phi
directory[21]=chi
directory[22]=psi
directory[23]=omega
# Printing out some possibly useful about your sub-job into the standard .out file. You'll see this file is
# named according to your job number and sub-job number in square brackets. $PBS_ARRAY_INDEX is your sub-job
# number (1-3 in this example). No need to change anything below.
echo "My job starts here"
date
uname -n
echo ' PBS_ARRAY_INDEX: ' $PBS_ARRAY_INDEX:wq
cd ${directory[$PBS_ARRAY_INDEX]}
echo ' directory: ' ${directory[$PBS_ARRAY_INDEX]}
pwd
echo '${directory[$PBS_ARRAY_INDEX]}.inp:' ${directory[$PBS_ARRAY_INDEX]}.inp
g09 < ${directory[$PBS_ARRAY_INDEX]}.com > ${directory[$PBS_ARRAY_INDEX]}.log
# Normal exit of script: clean up temporary files - Gaussian should do this anyway, but to be safe. Remove
# the Gaussian scratch directory where temporary files are written: /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# If abnormal termination (walltime or job killed) then subroutine cleanup_scratch will clear temporary files.
rm -rfv "$GAUSS_SCRDIR"
echo "Normal execution ending time:"
date
#!/bin/bash
# Any of these comment lines can remain in place and won't interfere with the script - or removed if you prefer.
# (1) These are standard type lines for PBS job specification. Modify them for your own requirements - choice of
# nodes; walltime; your directory on scratch for standard out and error files ; email of job starting,
# ending and errors.
#PBS -N My.job
#PBS -l select=2:ncpus=12:jobtype=westmere,place=free:group=nodetype:excl
#PBS -l walltime=0:20:00
#PBS -q workq
#PBS -m abe
#PBS -M MY_Email_Address
# End of comments (1) Make changes above.
# (2) The line below defined the job array. In this example there are 3 sub-jobs. Each sub-job is
# essentially an individual job but which uses the same settings such as node, walltime and email about job.
# "qstat -ftu username" is an appropriate way to check on your job array status, each subjob will be listed
# as [1], [2] etc. You can do a qdel on a sub-job, just use job number including square brackets.
#PBS -J 1-3
# (3) This is to clean up temporary files which are left behind if Gaussian crashes. No changes required below.
cleanup_scratch() {
echo "Deleting inside signal handler, meaning probably either hit walltime, or deleted the job using qdel"
cd "$PBS_O_WORKDIR"
rm -rfv "$GAUSS_SCRDIR"
echo "---"
echo "Signal handler ending time:"
date
exit 0
}
# Associate function "cleanup_scratch" with the TERM signal (and others), which is usually how jobs get killed.
trap 'cleanup_scratch' SIGTERM SIGHUP SIGINT SIGQUIT SIGFPE SIGKILL SIGALRM
# End of comments (2) No changes required above.
# (4) You can remove this first line below after you've run the script once, but won't be problem if it remains.
mkdir /lustre/SCRATCH5/gau/$USER
# (5) Please leave this line unchanged. Creating a subdirectory with your jobid for temporary Gaussian files
# which are removed after job. Previously everyone's Gaussian temporary files were written in /SCRATCH5/gau
mkdir /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# (6) Change to your work directory (usually more reliable than using "$PBS_O_WORKDIR"). Write your exact
# work directory in the format below using /lustre
cd /lustre/SCRATCH5/users/aslopis/Job_Array_Testing
# (7) These are arbitrary subdirectory names for each sub-job, please set up all your input files (.com and
# .chk) in this subdirectory and your output files will be written here. Hence each sub-job will be in
# separate directories and more clearly organised. The names are arbitrary, so you should name them in a
# meaningful way for you. Please note, the input files must also have the exact same name as the directory
# containing them so the script can then find and run using them. In the current example only 3 jobs are
# specified (PBS -J 1-3), but you can specify as many directories/sub-jobs as you like (more than 23 if
# desired). This bypasses the 10 queued job limit (counts as 1 job) but not problematic for the system - in
# fact better!
directory[1]=alpha
directory[1]=alpha
directory[2]=beta
directory[3]=gamma
directory[4]=delta
directory[5]=epsilon
directory[6]=eta
directory[7]=theta
directory[8]=iota
directory[9]=kappa
directory[10]=lamda
directory[11]=mu
directory[12]=nu
directory[13]=xi
directory[14]=omicron
directory[15]=pi
directory[16]=rho
directory[17]=sigma
directory[18]=tau
directory[19]=upsilon
directory[20]=phi
directory[21]=chi
directory[22]=psi
directory[23]=omega
# (8) Printing out some possibly useful about your sub-job into the standard .out file. You'll see this file
# is named according to your job number and sub-job number in square brackets. $PBS_ARRAY_INDEX is your
# sub-job number (1-3 in this example).
echo "My job starts here"
date
uname -n
echo ' PBS_ARRAY_INDEX: ' $PBS_ARRAY_INDEX
cd ${directory[$PBS_ARRAY_INDEX]}
echo ' directory: ' ${directory[$PBS_ARRAY_INDEX]}
pwd
echo '${directory[$PBS_ARRAY_INDEX]}.inp:' ${directory[$PBS_ARRAY_INDEX]}.inp
# (9) Please leave unchanged. This new module specifies the new directory for Gaussian variable GAUSS_SCRDIR
# for temporary files directory.
source /etc/profile.d/modules.sh
module add gaussian/g09.anton
source /opt/gridware/applications/gaussian/g09/bsd/g09.profile
# (10) Please leave unchanged. The directory/filename will be transferred automatically here.
LINDA=`cat $PBS_NODEFILE | uniq | tr '\n' "," | sed 's|,$||' `
echo linda: $LINDA
cat ${directory[$PBS_ARRAY_INDEX]}.com | sed "s/LINDA/$LINDA/" > temp$$.inp
g09 < temp$$.inp > ${directory[$PBS_ARRAY_INDEX]}.log
# (11) Leave following unchanged.
# Normal exit of script: clean up temporary files - Gaussian should do this anyway, but to be safe.
# Remove the Gaussian scratch directory where temporary files are written. It is
# /lustre/SCRATCH5/gau/$USER/$PBS_JOBID
# If abnormal termination (walltime or job killed) then function subroutine cleanup_scratch will clear
# temporary files.
rm -rfv "$GAUSS_SCRDIR"
echo "Normal execution ending time:"
date
Execute qg09 on login node and follow the prompts.
EXAMPLE1
Enter input file (with .com or .gjf extension)
test.gjf
Enter number of nodes on which to run job
1
Enter number of processors per node
12
Enter total walltime (hour:minute:second)
2:00:00
Enter name of queue on which to run job
workq
Generated pbs file for test
Do you wish to submit job to cluster (y/n)
y
EXAMPLE2
Enter input file (with .com or .gjf extension)
test.gjf
Enter number of nodes on which to run job
Enter number of processors per node
Enter total walltime (hour:minute:second)
Enter name of queue on which to run job
Generated pbs file for test
Do you wish to submit job to cluster (y/n)
y
This script handles PBS setup and submission. Also takes care of jobs running over more than one node with LINDA and does cleanup of GAU files when job is completed/killed.