wiki.chpc.ac.za

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== ANSYS/CFX ======

The CHPC has an installation of Ansys-CFD along with a limited license for academic use only.  The license covers use of the Fluent and CFX solvers, as well as the IcemCFD meshing code.  

===== Application Process =====

If you are a **full time** student or staff at an **academic** institution then you may request access to use Ansys-CFD on the CHPC clusters.  Send  your request along with motivation and description of the work in an email to ''helpdesk@chpc.ac.za''.

===== Installation =====

CFX versions are installed in ''/apps/chpc/compmech/CFD/ansys_inc/v???/CFX'', where ??? represents the software release version such as 242, for example. Similarly, IcemCFD is installed under ''/apps/chpc/compmech/CFD/ansys_inc/v???/icemcfd''.



===== Licensing =====

CHPC has academic licenses for AnsysCFD.  There are 25 "solver" processes and the license pool is not currently under heavy pressure.  There is a license resource management system to be used for Fluent, but it is not effective for older CFX versions.  

===== Running a CFX Job =====

On the CHPC clusters all simulations are submitted as jobs to the PBS Pro job scheduler which will assign your job to the appropriate queue and machine.

Example job script:

<file bash runCFX.qsub>
#!/bin/bash
#PBS -P projectid
#PBS -l select=4:ncpus=24:mpiprocs=24
#PBS -l walltime=2:00:00
#PBS -q normal
#PBS -V
#PBS -o /mnt/lustre/users/username/testCFX/test.out
#PBS -e /mnt/lustre/users/username/testCFX/test.err

export LM_LICENSE_FILE=1055@login1
export ANSYSLMD_LICENSE_FILE=1055@login1
#change to CFX case directory
cd /mnt/lustre/users/username/testCFX 
#force CFX to use ssh instead of rsh
export CFX5RSH=ssh 
# Select IntelMPI-2018 instead of 2021
export CFX5_IMPI_DIR=/apps/chpc/compmech/CFD/ansys_inc/v242/commonfiles/MPI/Intel/2018.3.222/linx64
cfx=/apps/chpc/compmech/CFD/ansys_inc/v242/CFX/bin/cfx5solve #path to cfx5 solver
$cfx -batch -def testCFX.def -par-dist $(tr "\n" "," <$PBS_NODEFILE | rev | cut -c2- | rev) > runcfx.log
</file>
===== Different methods of uploading a simulation =====

==== Build and test locally, upload .def file ====
The "standard" process assumes that the user already has a local license for the software.  
   * Mesh and pre-process the simulation as usual for a local simulation.
   * Test it locally to ensure that everything works properly. Be cautious about absolute path file names.
   * Compress the def file with gzip.
   * Upload to CHPC using either scp or rsync.  The advantage of rsync is that the transfer can be made persistent, to prevent network communication glitches from killing the file transfer.

==== Build and test locally, upload geometry and script files only, mesh and pre-process remotely ====
If your simulations files are too large, or your internet connection too slow, consider transferring geometry and script files only.  This will require careful scripting and testing, but is certainly practical.
   * Neither ANSYS-Mesh nor Gambit is available on the CHPC system, but ICEMCFD is.
   * If using IcemCFD, transfer the Icem .prj, .tin, .fbc and .blk (if using hexa) files, along with a recorded Icem script file for generating the mesh and exporting the file in CFX format.  Watch for absolute path names in the script file.  Run IcemCFD with the -batch -script options to create the mesh.  A more comprehensive CFX script will be required to import the mesh and pre-process the case. **Test locally!**
   * If your internet connection is too slow to permit easy case uploading, it will also be far too slow for downloading the results files.  Consider generating post-processing images "on the fly", or alternatively exporting only surface data on completion of the simulation.

===== General tips and advice =====
   * Give some thought to the resources being requested.  Partitioning a simulation too finely will not necessarily speed it up as expected.  In general, less than 100 000 cells per process is seldom justified.
   * A request for a smaller number of cores may result in the job launching earlier, resulting in reduced turn-around time, even if the job takes longer to run.
   * Monitoring convergence of batch jobs can be painful but necessary.
   * Monitor files (such as cd or cl files) can be plotted with gnuplot even if no GUI is available.  On a slow connection, consider using gnuplot with ''set term dumb'' to get funky 1970's style ASCII graphics.
   *  If you need to submit a large number of small jobs, when doing a parametric study, for example, please use Job Arrays.  Refer to the PBS-Pro guide at http://wiki.chpc.ac.za/quick:pbspro for guidance on how to set this up.
wiki.chpc.ac.za

User Tools

Site Tools

Page Tools
