Differences

This shows you the differences between two versions of the page.

--- howto:palabos [2022/05/13 09:58]
ccrosby
+++ howto:palabos [2022/05/13 14:59] (current)
ccrosby [Instructions]
@@ Line 1: / Line 1: @@
 ======  Using the Lattice-Boltzman code Palabos  ======
 The following instructions are courtesy of Dr. Muaaz Bhamjee of the University of Johannesburg.
+===== Introduction =====
 Palabos is an open source CFD package based on the Lattice Boltzmann Method (LBM). Palabos is effectively a C++ library that can be used to develop CFD applications. More information can be obtained at the [[https://palabos.unige.ch/|Palabos website]].
@@ Line 6: / Line 8: @@
 Refer to the following book for an introduction to the LBM:
 M. C. Sukop and J. D. T. Thorne, Lattice Boltzmann Modelling: An Introduction for Geoscientists and Engineers, 2nd ed. Miami. FL: Springer-Verlag, 2006. It can be obtained [[https://doi.org/10.1007/978-3-540-27982-2|here]].
+==== Instructions ====
 To use Palabos on the CHPC you must first install the libraries to your user directory. To do so, go to the [[https://gitlab.com/unigespc/palabos| Palabos git repository]]  and download the latest version as a tar file. Transfer this file to your user directory and extract the tar file. You can rename the install folder (this is optional)
-//Assuming you have downloaded the file to your user directory, and you have logged onto the Lengau cluster//
+Assuming you have downloaded the file to your user directory, and you have logged onto the Lengau cluster
+<code>
 cd /mnt/lustre/users/username/
 tar -xzvf palabos-master.tar.gz
 //This is the optional step of renaming the palabos folder//
 mv palabos-master palabos
+</code>
 The next step is to load the cmake module
+<code>
 module load chpc/compmech/cmake/3.15.0
+</code>
 To demonstrate how to compile and run a Palabos simulation, I will provide the scripts and instructions for the aneurysm example included in the palabos library. First change to the example build directory (assuming the Palabos install was renamed to palabos):
+<code>
 cd /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build
+</code>
+The next step is to compile the code. This can be run as a job with the script given below, or in an [[https://wiki.chpc.ac.za/howto:tipsandtricks#using_an_interactive_pbs_session|interactive PBS session]]. Note that the OpenFOAM environment is invoked as it contains all the dependencies for Palabos apart from cmake. The cmake module is added in the job script given below.  If you are confident that the build will work first time, you can use this script, otherwise do it in an interactive session, following the same steps:
+  - Get into the right directory
+  - Source the OpenFOAM environment
+  - Load the cmake module
+  - Run ''cmake ../''
+  - Run ''make''
+=== Compile script ===
+<file bash PalabosCompile.pbs>
+#!/bin/bash
+#PBS -P projectcode
+#PBS -l select=1:ncpus=24:mpiprocs=24:nodetype=haswell_reg
+#PBS -q smp
+#PBS -l walltime=01:00:00
+#PBS -o /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build/stdout
+#PBS -e /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build/stderr
+#PBS -m abe
+#PBS -M username@email.com
+### Source the openFOAM environment (palabos needs this for the dependencies):
+. /apps/chpc/compmech/CFD/OpenFOAM/OF2112
+### load the cmake module
+module add chpc/compmech/cmake/3.15.0
+##### Running commands
+# Set this environment variable explicitly.
+export PBS_JOBDIR=/mnt/lustre/users/username/palabos/examples/showCases/aneurysm/build
+# Explicitly change to the job directory
+cd $PBS_JOBDIR
+cmake .. > cmake.out
+make > make.out
+</file>
+=== Running the example case ===
+To run the example you must change back one directory level to this directory: <code> /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/ </code>
+If you have not run this example before, first uncompress the geometry file:
+<code> tar xzf aneurysm.tgz </code>
+If your OpenFOAM environment is still valid, you can run your aneurysm example on a single node with the following instruction:
+<code>  mpirun -np 24 ./aneurysm param.xml  </code>
+To run the code as a PBS batch job over several nodes, you can use this script:
+<file bash runPalabos.qsub>
+#!/bin/bash
+#PBS -P projectcode
+#PBS -l select=5:ncpus=24:mpiprocs=24
+#PBS -q normal
+#PBS -l walltime=4:00:00
+#PBS -o /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/stdout
+#PBS -e /mnt/lustre/users/username/palabos/examples/showCases/aneurysm/stderr
+#PBS -m abe
+#PBS -M username@email.com
+### Source the openFOAM environment (palabos needs this for the dependencies):
+. /apps/chpc/compmech/CFD/OpenFOAM/OF2112
+### load the cmake module
+module add chpc/compmech/cmake/3.15.0
+##### Running commands
+# Set this environment variable explicitly.
+export PBS_JOBDIR=/mnt/lustre/users/username/palabos/examples/showCases/aneurysm/
+# Explicitly change to the job directory
+cd $PBS_JOBDIR
+nproc=`cat $PBS_NODEFILE | wc -l`
+###Run the code using MPI
+exe=aneurysm
+mpirun -iface ib0 -np $nproc $exe param.xml > run.out
+</file>
+=== Post-processing ===
+Palabos does not come with built-in post-processing tools.  However, it makes use of some [[https://imagemagick.org/index.php|ImageMagick]] utilities and can also produce output files in VTK format.  On Lengau there are two ImageMagick modules available:
+<code>
+chpc/compmech/ImageMagick/7.0.5
+chpc/compmech/ImageMagick/image-magick-7.0.5-9-gcc-8.3.0-fvnxz3q
+</code>
-The next step is to compile the code. This must be run as a job with the below script. Note that the OpenFOAM environment is invoked as it contains all the dependencies for Palabos apart from cmake. With the cmake module loaded, it will be added through the compile script:
+Load one of these modules if you need ImageMagick.
+VTK files can be displayed with [[https://paraview.org/|Paraview]].  This Wiki has a dedicated page for [[howto:paraview|running Paraview]] on the cluster.

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools