User Tools

Site Tools


howto:cfx

ANSYS/CFX

The CHPC has an installation of Ansys-CFD along with a limited license for academic use only. The license covers use of the Fluent and CFX solvers, as well as the IcemCFD meshing code.

Application Process

If you are a full time student or staff at an academic institution then you may request access to use Ansys-CFD on the CHPC clusters. Send your request along with motivation and description of the work in an email to helpdesk@chpc.ac.za.

Installation

CFX version 5 is installed in /apps/chpc/compmech/CFD/ansys_inc/v170/CFX. IcemCFD is installed under /apps/chpc/compmech/CFD/ansys_inc/v170/icemcfd.

Licensing

CHPC has academic licenses for AnsysCFD. There are 5 “solver” processes, available as aa_r_cfd and 500 “HPC” licenses, available as aa_r_hpc. There is a license resource management system. If you request license resources (as in these example scripts), the scheduler will check for license availability before starting a job. License unavailability will result in the job being held back until the necessary licenses have become available. Although use of the license resource request is not mandatory, its use is strongly recommended. If you do not use the license resource requests, the job will fail if no licenses are available. A single aa_r_cfd license is required to start the solver, and includes up to 4 HPC licenses. Therefore you should request ($nproc-4) aa_r_hpc licenses.

Running a CFX Job

On the CHPC clusters all simulations are submitted as jobs to the PBS Pro job scheduler which will assign your job to the appropriate queue and machine.

Example job script:

runCFX.qsub
#!/bin/bash
#PBS -P projectid
#PBS -l select=4:ncpus=24:mpiprocs=24:mem=24GB:nodetype=haswell_reg
#PBS -l walltime=2:00:00
#PBS -q normal
#PBS -m be
#PBS -V
#PBS -o /home/username/testCFX/test.out
#PBS -e /home/username/testCFX/test.err
 
cd /home/username/testCFX #change to CFX case directory
export CFX5RSH=ssh #force CFX to use ssh instead of rsh
cfx=/apps/chpc/compmech/CFD/ansys_inc/v150/CFX/bin/cfx5solve #path to cfx5 solver
$cfx -batch -def testCFX.def -par-dist $(tr "\n" "," <$PBS_NODEFILE)

Different methods of uploading a simulation

Build and test locally, upload .def file

The “standard” process assumes that the user already has a local license for the software.

  • Mesh and pre-process the simulation as usual for a local simulation.
  • Test it locally to ensure that everything works properly. Be cautious about absolute path file names.
  • Compress the def file with gzip.
  • Upload to CHPC using either scp or rsync. The advantage of rsync is that the transfer can be made persistent, to prevent network communication glitches from killing the file transfer.

Build and test locally, upload geometry and script files only, mesh and pre-process remotely

If your simulations files are too large, or your internet connection too slow, consider transferring geometry and script files only. This will require careful scripting and testing, but is certainly practical.

  • Neither ANSYS-Mesh nor Gambit is available on the CHPC system, but ICEMCFD is.
  • If using IcemCFD, transfer the Icem .prj, .tin, .fbc and .blk (if using hexa) files, along with a recorded Icem script file for generating the mesh and exporting the file in CFX format. Watch for absolute path names in the script file. Run IcemCFD with the -batch -script options to create the mesh. A more comprehensive CFX script will be required to import the mesh and pre-process the case. Test locally!
  • If your internet connection is too slow to permit easy case uploading, it will also be far too slow for downloading the results files. Consider generating post-processing images “on the fly”, or alternatively exporting only surface data on completion of the simulation.

General tips and advice

  • Give some thought to the resources being requested. Partitioning a simulation too finely will not necessarily speed it up as expected. In general, less than 100 000 cells per process is seldom justified.
  • A request for a smaller number of cores may result in the job launching earlier, resulting in reduced turn-around time, even if the job takes longer to run.
  • Monitoring convergence of batch jobs can be painful but necessary.
  • Monitor files (such as cd or cl files) can be plotted with gnuplot even if no GUI is available. On a slow connection, consider using gnuplot with set term dumb to get funky 1970's style ASCII graphics.
  • If you need to submit a large number of small jobs, when doing a parametric study, for example, please use Job Arrays. Refer to the PBS-Pro guide at http://wiki.chpc.ac.za/quick:pbspro for guidance on how to set this up.
/var/www/wiki/data/pages/howto/cfx.txt · Last modified: 2017/09/11 13:57 by ccrosby