User Tools

Site Tools


Using MFIX at the CHPC

Instructional video: MFix at the CHPC.


MFIX is an open-source multiphase flow solver written in FORTRAN 90. It is used for simulating fluid-solid systems such as fluidized beds. As it appears to have numerous potential uses in chemical engineering and mineral processing applications, a limited level of support for the use of the package is now available at the CHPC.

Alternatively, users may download the code and install it in their home directory. A one-time free registration is required prior to downloading the source code. To register, go to the MFIX website at, click on “Register” at the bottom of the home page or go directly to

This wiki page details those issues specific to running MFIX on the Lengau cluster at CHPC. For a more general explanation of the use of MFIX, consult the documentation and example cases included with the source tarball.

Modern MFix

In recent years, MFix has been substantially modernised. It can be used in two ways:

  1. Through a very slick Python-based GUI.
  2. By customizing the solver and running from the command line.

However, even when using the customized command line approach, a Python script is still used for building the code. The following instructions apply to MFix-23.2, which has been set up to facilitate both of these approaches.


A Miniconda environment for MFix-23.2 has been configured, and is installed in the directory /home/apps/chpc/compmech/MFix/miniconda3/envs/mfix-23.2. These are the required steps:

Step 1: Source the conda shell script

This command can be issued from the command line or be placed in your $HOME/.bashrc file for interactive work. When running in non-interactive batch mode through PBS, it should be placed in your PBS script.

source /home/apps/chpc/compmech/MFix/miniconda3/etc/profile.d/

Step 2: Load the appropriate mfix module

Like the previous instruction, this can be given from the command line or placed in your $HOME/.bashrc file for interactive work, or placed in the PBS job script.

module load chpc/compmech/mfix/23.2

Step 3: Activate the conda environment

This step is necessary if you want to use the GUI-based process or if you want to compile a custom solver. It is not necessary if you simply want to run an already-compiled custom solver.

conda activate mfix-23.2

Step 4: Compile a custom solver

The standard mfixsolver is already in your path. To build a custom solver that incorporates user coding and/or parallelisation, use the provided build_mfixsolver procedure with the appropriate options. This example provides both smp (OpenMP) and dmp (MPI) parallelisation and is compiled using 4 threads.

build_mfixsolver --batch --smp --dmp -j 4

Step 5: Run the solver

If you are going to run a custom solver, first remove old out files, then set your required number of OpenMP threads, if appropriate, and run. This example is for smp only.


Example job script

### Request a single node for 4 MPI processes and 2 OpenMP threads per MPI process
#PBS -l select=1:ncpus=8:mpiprocs=4
#PBS -P MECH1234
#PBS -l walltime=02:00:00
#PBS -q serial
#PBS -o /home/jblogs/lustre/mfix-23.2/tests/fluid/FLD03/mfix.out
#PBS -e /home/jblogs/lustre/mfix-23.2/tests/fluid/FLD03/mfix.err
### Change directory to a typical test case that comes with MFix.  Obviously use your own directory. 
cd /home/jblogs/lustre/mfix-23.2/tests/fluid/FLD03
### Prepare the Miniconda environment
source /home/apps/chpc/compmech/MFix/miniconda3/etc/profile.d/
### Get the following into your path:
###   1. The standard MFix installation
###   2. gcc-8.3.0
###   3. mpich-3.3
module load chpc/compmech/mfix/23.2
### Activate the mfix conda environment
conda activate mfix-23.2
### Build your custom solver for both OpenMP and MPI.  Use 4 threads for the compile.
build_mfixsolver --batch --smp --dmp -j 4
### We no longer need the mfix conda environment.  Deactivate it.
conda deactivate
### Set up 2 OpenMP threads per MPI process
### The MFix test case comes with a script to run the case.  This script contains the mpirun command which 
###  can be edited.  In this particular case it had te be edited to remove two options pertaining to 
###  running as root and oversubscribing cores.  The mpirun command looks like this:
###   mpirun -np 4 ./mfixsolver -f mfix.dat nodesi=2 nodesj=2  
###  and can also be used below instead of the the script.
./ | tee mfixsolver.out

Interactive MFix

With the environment set up by these instructions:

source /home/apps/chpc/compmech/MFix/miniconda3/etc/profile.d/
module load chpc/compmech/mfix/23.2
conda activate mfix-23.2

it is also possible to run the interactive version of MFix with its Python-based GUI. However, there are two main problems:

  1. Doing graphics with X-forwarding is slow
  2. Compute nodes do not have GPUs that can handle OpenGL graphics

These two problems are easy to overcome. Use VNC to work with a virtual desktop and parallel Mesa software rendering to provide OpenGL graphics. Please read this page:

Working with older versions

Building MFIX

MFIX generates a new executable for each case, which is copied in to the case directory. First cd to the case directory. Next, add modules for your choice of gcc version and the corresponding version of MPI, for instance:

module add gcc/5.1.0 chpc/openmpi/gcc/65/1.8.8-gcc5.1.0

Finally, the mfix executable is built and copied to the current case directory by the following command:

sh /apps/chpc/compmech/CFD/MFIX/mfix/model/make_mfix

if using the system mfix, or

sh <mfix_install_dir>/model/make_mfix

for other install locations.

This script prompts the user to specify a number of compile-time options, in particular the compiler used, the desired level of optimisation and the type of parallelisation used. The default compiler is gfortran. The Intel fortran compiler may also be used, but this has not yet been tested on the Sun cluster. The parallelisation options are serial, parallel shared-memory, parallel distributed memory, or parallel hybrid (shared and distributed).

At present, parallel distributed memory appears to work best; reasonable scaling seems to be obtained. Shared memory parallel works for many cases, but does not yield a significant improvement for the cases tested to date. Hybrid parallelization is currently under development, and it is best avoided at present. It should be noted that use of the Johnson and Jackson partial slip boundary condition (BC_JJ in mfix.dat file) causes a crash for all methods of parallelisation (although it works for serial computations).

Setting up and submitting a job

The input for an MFIX case consists of an mfix.dat file, a text file which defines most or all of the properties of case (geometry, boundary and initial conditions, choice of turbulence and friction models, and so on), as well as any fortran source files containing user-extensions to the standard MFIX solver, and any additional optional files describing geometry. In most cases the mfix.dat file is sufficient, and as this file is relatively small, it may be uploaded using scp.

Below is an example of a simple PBS submit script for an mfix job.

  #PBS -P projectid
  #PBS -l select=3:ncpus=24:mpiprocs=24:mem=12GB:nodetype=haswell_reg 
  #PBS -q normal
  #PBS -l walltime=01:00:00
  #PBS -o /mnt/lustre/users/username/job01/stdout
  #PBS -e /mnt/lustre/users/username/job01/stderr
  #PBS -m abe
  #PBS -M
  module add chpc/compmech/mfix/20.1.0
  nproc=`cat $PBS_NODEFILE | wc -l`
  mpirun -np $nproc -machinefile $PBS_NODEFILE $exe -parallel >"coarse.log" 2>&1

Remember to edit or add the following line to your mfix.dat file:


where NX, NY and NZ should be replaced by the number of partitions along each physical principle axis of the model, so that NX*NY*NZ is the total number of cores requested in the submit script. In general, it is best to choose NX, NY and NZ such that largest number of partitions occur along the axis/axes corresponding roughly with the average flow direction.


Please be advised that there is now also a version 20.3.0 in /apps/chpc/compmech/CFD/MFIX/20.3.0. There is a source script in that directory, setMFix, which will load the correct gcc and MPI modules and append the $PATH to make the executable mfixsolver available. This binary supports both thread-level (OpenMP) and distributed memory (MPI) parallel. It uses mpich, and it is necessary to use the -iface ib0 option to ensure the use of the Infiniband network. A machinefile should not be used. The following job script should work, but has not been tested:

  #PBS -P projectid
  #PBS -l select=3:ncpus=24:mpiprocs=24:mem=12GB:nodetype=haswell_reg 
  #PBS -q normal
  #PBS -l walltime=01:00:00
  #PBS -o /mnt/lustre/users/username/job01/stdout
  #PBS -e /mnt/lustre/users/username/job01/stderr
  #PBS -m abe
  #PBS -M
  . /apps/chpc/compmech/CFD/MFIX/20.3.0/setMFix
  nproc=`cat $PBS_NODEFILE | wc -l`
  mpirun -iface ib0 -np $nproc $exe -parallel >"coarse.log" 2>&1


Please be advised that there is now also a version 22.2.2 in /home/apps/chpc/compmech/MFix-22.2.2. The executable mfixsolver can be added to your path with the following module command:

module load chpc/compmech/mfix/22.2.2

Please note that this has been compiled with gfortran-9.2.0 and MPICH-4.0. When running in parallel, add the option -iface ib0 to ensure that the high speed Infiniband network is used. This version of MPICH has been compiled with support for PBS, therefore a hosts or machine file is not needed. This command line should work:

mpirun -iface ib0 -np $nproc mfixsolver -f myinputfile.mfx


Mfix outputs a .RES file in the case directory, in addition to optional VTK files representing the solution. Both types of files may be opened using Paraview or a similar VTK viewer.

If postprocessing is to be done on CHPC hardware via the network, the following link may be helpful to get decent performance: Remote OpenGL visualization with TurboVNC and VirtualGL

If the postprocessing is to be performed on the user's local machine, the entire contents of the MFIX case directory should be copied to the user's machine using rsynch or scp.

/app/dokuwiki/data/pages/howto/mfix.txt · Last modified: 2023/09/15 12:36 by ccrosby