Differences

This shows you the differences between two versions of the page.

--- howto:mfix [2025/04/16 11:57]
ccrosby
+++ howto:mfix [2025/04/16 12:28] (current)
ccrosby [MFix-Exa]
@@ Line 18: / Line 18: @@
 However, even when using the customized command line approach, a Python script is still used for building the code.  The following instructions apply from MFix-23.2 onwards, which have been set up to facilitate both of these approaches.
-==== MFix-exa ====
+==== MFix-Exa ====
-Further to the theme of modernised MFix, is the development of [[https://mfix.netl.doe.gov/products/mfix-exa/|MFix-exa]], a high-performance, massively parallel, machine agnostic multiphase flow code for the exascale computing era. MFIX-Exa is built using the AMReX framework which provides iterators, linear solvers, parallel communication routines, and other utilities that support highly efficient operations on structured grid and particle data. MFIX-Exa performance portability is inherited from AMReX’s flexible MPI+X hybrid parallelism strategy allowing MFIX-Exa to offload to Nvidia, AMD, and Intel GPU accelerators.
+Further to the theme of modernised MFix, is the development of [[https://mfix.netl.doe.gov/products/mfix-exa/|MFix-Exa]], a high-performance, massively parallel, machine agnostic multiphase flow code for the exascale computing era. MFIX-Exa is built using the AMReX framework which provides iterators, linear solvers, parallel communication routines, and other utilities that support highly efficient operations on structured grid and particle data. MFIX-Exa performance portability is inherited from AMReX’s flexible MPI+X hybrid parallelism strategy allowing MFIX-Exa to offload to Nvidia, AMD, and Intel GPU accelerators.  Due to a scandalous shortage of GPUs, the GPU version is not available at the CHPC.  The latest installed version is 2025-04.1, installed in the directory ''/home/apps/chpc/compmech/mfix-exa-25.04.1''  The executable is installed in the subdirectory ''exec''.  MFix-exa-25.04.1 on Lengau has been compiled with Intel's OneAPI-2023.2.0 compiler and mpich-4.2.2 with support for OpenMP.  The "superbuild" executable is called ''mfix3d.intel-llvm.MPI.OMP.ex'', but is also accessible with a symbolic link ''mfix-exa''.  To set up the environment, source the following script:
+<code>.  /home/apps/chpc/compmech/mfix-exa-25.04.1/setMfixIntel </code>
+=== Example job script for MFix-Exa ===
+<file bash runMFix-exa.pbs>
+#!/bin/bash
+### Request two compute nodes, each with 12 MPI processes and 2 OpenMP threads per MPI process
+#PBS -l select=2:ncpus=24:mpiprocs=12
+#PBS -P MECH1234
+#PBS -l walltime=02:00:00
+#PBS -q normal
+#PBS -o /home/jblogs/lustre/mfix-exa/tests/fluid/FLD03/mfix.stdout
+#PBS -e /home/jblogs/lustre/mfix-exa/tests/fluid/FLD03/mfix.stderr
+### Change directory to a typical test case that comes with MFix.  Obviously use your own directory.
+cd /home/jblogs/lustre/mfix-exa/tests/fluid/FLD03
+### Set up the required environment
+.  /home/apps/chpc/compmech/mfix-exa-25.04.1/setMfixIntel
+### Set up 2 OpenMP threads per MPI process
+export OMP_NUM_THREADS=2
+### Find the number of MPI processes
+nproc=`cat $PBS_NODEFILE | wc -l`
+mpirun -iface ib0 -np $nproc mfix-exa inputs > mfix.out
+</file>
 ==== MFix-24.2.3 ====

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools