Differences

This shows you the differences between two versions of the page.

--- howto:palabos [2022/05/13 10:10]
ccrosby [Instructions]
+++ howto:palabos [2022/05/13 14:59] (current)
ccrosby [Instructions]
@@ Line 22: / Line 22: @@
 mv palabos-master palabos
 </code>
 The next step is to load the cmake module
+<code>
 module load chpc/compmech/cmake/3.15.0
+</code>
 To demonstrate how to compile and run a Palabos simulation, I will provide the scripts and instructions for the aneurysm example included in the palabos library. First change to the example build directory (assuming the Palabos install was renamed to palabos):
+<code>
 cd /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build
+</code>
+The next step is to compile the code. This can be run as a job with the script given below, or in an [[https://wiki.chpc.ac.za/howto:tipsandtricks#using_an_interactive_pbs_session|interactive PBS session]]. Note that the OpenFOAM environment is invoked as it contains all the dependencies for Palabos apart from cmake. The cmake module is added in the job script given below.  If you are confident that the build will work first time, you can use this script, otherwise do it in an interactive session, following the same steps:
+  - Get into the right directory
+  - Source the OpenFOAM environment
+  - Load the cmake module
+  - Run ''cmake ../''
+  - Run ''make''
+=== Compile script ===
+<file bash PalabosCompile.pbs>
+#!/bin/bash
+#PBS -P projectcode
+#PBS -l select=1:ncpus=24:mpiprocs=24:nodetype=haswell_reg
+#PBS -q smp
+#PBS -l walltime=01:00:00
+#PBS -o /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build/stdout
+#PBS -e /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build/stderr
+#PBS -m abe
+#PBS -M username@email.com
+### Source the openFOAM environment (palabos needs this for the dependencies):
+. /apps/chpc/compmech/CFD/OpenFOAM/OF2112
+### load the cmake module
+module add chpc/compmech/cmake/3.15.0
+##### Running commands
+# Set this environment variable explicitly.
+export PBS_JOBDIR=/mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build
+# Explicitly change to the job directory
+cd $PBS_JOBDIR
+cmake .. > cmake.out
+make > make.out
+</file>
+=== Running the example case ===
+To run the example you must change back one directory level to this directory: <code> /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/ </code>
+If you have not run this example before, first uncompress the geometry file:
+<code> tar xzf aneurysm.tgz </code>
+If your OpenFOAM environment is still valid, you can run your aneurysm example on a single node with the following instruction:
+<code>  mpirun -np 24 ./aneurysm param.xml  </code>
+To run the code as a PBS batch job over several nodes, you can use this script:
+<file bash runPalabos.qsub>
+#!/bin/bash
+#PBS -P projectcode
+#PBS -l select=5:ncpus=24:mpiprocs=24
+#PBS -q normal
+#PBS -l walltime=4:00:00
+#PBS -o /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/stdout
+#PBS -e /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/stderr
+#PBS -m abe
+#PBS -M username@email.com
+### Source the openFOAM environment (palabos needs this for the dependencies):
+. /apps/chpc/compmech/CFD/OpenFOAM/OF2112
+### load the cmake module
+module add chpc/compmech/cmake/3.15.0
+##### Running commands
+# Set this environment variable explicitly.
+export PBS_JOBDIR=/mnt/lustre/users/username/palabos/examples/showCases/aneurysm/
+# Explicitly change to the job directory
+cd $PBS_JOBDIR
+nproc=`cat $PBS_NODEFILE | wc -l`
+###Run the code using MPI
+exe=aneurysm
+mpirun -iface ib0 -np $nproc $exe param.xml > run.out
+</file>
+=== Post-processing ===
+Palabos does not come with built-in post-processing tools.  However, it makes use of some [[https://imagemagick.org/index.php|ImageMagick]] utilities and can also produce output files in VTK format.  On Lengau there are two ImageMagick modules available:
+<code>
+chpc/compmech/ImageMagick/7.0.5
+chpc/compmech/ImageMagick/image-magick-7.0.5-9-gcc-8.3.0-fvnxz3q
+</code>
-The next step is to compile the code. This must be run as a job with the below script. Note that the OpenFOAM environment is invoked as it contains all the dependencies for Palabos apart from cmake. With the cmake module loaded, it will be added through the compile script:
+Load one of these modules if you need ImageMagick.
+VTK files can be displayed with [[https://paraview.org/|Paraview]].  This Wiki has a dedicated page for [[howto:paraview|running Paraview]] on the cluster.

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools