| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
howto:mfix [2025/04/16 12:11]
ccrosby [MFix-exa] |
howto:mfix [2025/04/16 12:28] (current)
ccrosby [MFix-Exa] |
| However, even when using the customized command line approach, a Python script is still used for building the code. The following instructions apply from MFix-23.2 onwards, which have been set up to facilitate both of these approaches. | However, even when using the customized command line approach, a Python script is still used for building the code. The following instructions apply from MFix-23.2 onwards, which have been set up to facilitate both of these approaches. |
| |
| ==== MFix-exa ==== | ==== MFix-Exa ==== |
| Further to the theme of modernised MFix, is the development of [[https://mfix.netl.doe.gov/products/mfix-exa/|MFix-exa]], a high-performance, massively parallel, machine agnostic multiphase flow code for the exascale computing era. MFIX-Exa is built using the AMReX framework which provides iterators, linear solvers, parallel communication routines, and other utilities that support highly efficient operations on structured grid and particle data. MFIX-Exa performance portability is inherited from AMReX’s flexible MPI+X hybrid parallelism strategy allowing MFIX-Exa to offload to Nvidia, AMD, and Intel GPU accelerators. Due to a scandalous shortage of GPUs, the GPU version is not available at the CHPC. The latest installed version is 2025-04.1, installed in the directory ''/home/apps/chpc/compmech/mfix-exa-25.04.1'' The executable is installed in the subdirectory ''exec''. MFix-exa-25.04.1 on Lengau has been compiled with Intel's OneAPI-2023.2.0 compiler and mpich-4.2.2 with support for OpenMP. The "superbuild" executable is called ''mfix3d.intel-llvm.MPI.OMP.ex'', but is also accessible with a symbolic link ''mfix-exa''. To set up the environment, source the following script: | Further to the theme of modernised MFix, is the development of [[https://mfix.netl.doe.gov/products/mfix-exa/|MFix-Exa]], a high-performance, massively parallel, machine agnostic multiphase flow code for the exascale computing era. MFIX-Exa is built using the AMReX framework which provides iterators, linear solvers, parallel communication routines, and other utilities that support highly efficient operations on structured grid and particle data. MFIX-Exa performance portability is inherited from AMReX’s flexible MPI+X hybrid parallelism strategy allowing MFIX-Exa to offload to Nvidia, AMD, and Intel GPU accelerators. Due to a scandalous shortage of GPUs, the GPU version is not available at the CHPC. The latest installed version is 2025-04.1, installed in the directory ''/home/apps/chpc/compmech/mfix-exa-25.04.1'' The executable is installed in the subdirectory ''exec''. MFix-exa-25.04.1 on Lengau has been compiled with Intel's OneAPI-2023.2.0 compiler and mpich-4.2.2 with support for OpenMP. The "superbuild" executable is called ''mfix3d.intel-llvm.MPI.OMP.ex'', but is also accessible with a symbolic link ''mfix-exa''. To set up the environment, source the following script: |
| <code> . /home/apps/chpc/compmech/mfix-exa-25.04.1/setMfixIntel </code> | <code>. /home/apps/chpc/compmech/mfix-exa-25.04.1/setMfixIntel </code> |
| |
| | === Example job script for MFix-Exa === |
| | <file bash runMFix-exa.pbs> |
| | #!/bin/bash |
| | ### Request two compute nodes, each with 12 MPI processes and 2 OpenMP threads per MPI process |
| | #PBS -l select=2:ncpus=24:mpiprocs=12 |
| | #PBS -P MECH1234 |
| | #PBS -l walltime=02:00:00 |
| | #PBS -q normal |
| | #PBS -o /home/jblogs/lustre/mfix-exa/tests/fluid/FLD03/mfix.stdout |
| | #PBS -e /home/jblogs/lustre/mfix-exa/tests/fluid/FLD03/mfix.stderr |
| | |
| | ### Change directory to a typical test case that comes with MFix. Obviously use your own directory. |
| | cd /home/jblogs/lustre/mfix-exa/tests/fluid/FLD03 |
| | |
| | ### Set up the required environment |
| | . /home/apps/chpc/compmech/mfix-exa-25.04.1/setMfixIntel |
| | |
| | ### Set up 2 OpenMP threads per MPI process |
| | export OMP_NUM_THREADS=2 |
| | |
| | ### Find the number of MPI processes |
| | nproc=`cat $PBS_NODEFILE | wc -l` |
| | |
| | mpirun -iface ib0 -np $nproc mfix-exa inputs > mfix.out |
| | </file> |
| |
| |
