Differences

This shows you the differences between two versions of the page.

--- howto:cfx [2014/05/08 14:52]
agill created
+++ howto:cfx [2025/02/21 12:45] (current)
ccrosby [Running a CFX Job]
@@ Line 1: / Line 1: @@
-====== ANSYS/Fluent ======
+====== ANSYS/CFX ======
 The CHPC has an installation of Ansys-CFD along with a limited license for academic use only.  The license covers use of the Fluent and CFX solvers, as well as the IcemCFD meshing code.
@@ Line 5: / Line 5: @@
 ===== Application Process =====
-If you are a **full time** student or staff at an **academic** institution then you may request access to use Ansys-CFD on the CHPC clusters.  Send  your request along with motivation and description of the work in an email to '''helpdesk@chpc.ac.za'''.
+If you are a **full time** student or staff at an **academic** institution then you may request access to use Ansys-CFD on the CHPC clusters.  Send  your request along with motivation and description of the work in an email to ''helpdesk@chpc.ac.za''.
 ===== Installation =====
-Fluent version  14.5.7 is installed in '''/opt/gridware/applications/ANSYS/ansys_inc/v145/fluent'''. There is also a slightly older version 14.5 which is installed in '''/opt/gridware/applications/ANSYS/Fluent'''.  Ansys CFX and IcemCFD are installed under '''/opt/gridware/applications/ANSYS/ansys_inc/v145'''.
+CFX versions are installed in ''/apps/chpc/compmech/CFD/ansys_inc/v???/CFX'', where ??? represents the software release version such as 242, for example. Similarly, IcemCFD is installed under ''/apps/chpc/compmech/CFD/ansys_inc/v???/icemcfd''.
-Version 15.0 editions of the Ansys software are installed under '''/opt/gridware/applications/ANSYS/ansys_inc/v150'''.
 ===== Licensing =====
-CHPC has academic licenses for AnsysCFD.  There are 5 "solver" processes, available as aa_r_cfd and 500 "HPC" licenses, available as aa_r_hpc.  There is a license resource management system.  If you request license resources (as in these example scripts), the scheduler will check for license availability before starting a job.  License unavailability will result in the job being held back until the necessary licenses have become available.  Although use of the license resource request is not mandatory, its use is strongly recommended.  If you do not use the license resource requests, the job will fail if no licenses are available.  A single aa_r_cfd license is required to start the solver, and includes up to 4 HPC licenses.  Therefore you should request ($nproc-4) aa_r_hpc licenses.
+CHPC has academic licenses for AnsysCFD.  There are 25 "solver" processes and the license pool is not currently under heavy pressure.  There is a license resource management system to be used for Fluent, but it is not effective for older CFX versions.
+===== Running a CFX Job =====
+On the CHPC clusters all simulations are submitted as jobs to the PBS Pro job scheduler which will assign your job to the appropriate queue and machine.
+Example job script:
+<file bash runCFX.qsub>
+#!/bin/bash
+#PBS -P projectid
+#PBS -l select=4:ncpus=24:mpiprocs=24
+#PBS -l walltime=2:00:00
+#PBS -q normal
+#PBS -V
+#PBS -o /mnt/lustre/users/username/testCFX/test.out
+#PBS -e /mnt/lustre/users/username/testCFX/test.err
+export LM_LICENSE_FILE=1055@login1
+export ANSYSLMD_LICENSE_FILE=1055@login1
+#change to CFX case directory
+cd /mnt/lustre/users/username/testCFX
+#force CFX to use ssh instead of rsh
+export CFX5RSH=ssh
+# Select IntelMPI-2018 instead of 2021
+export CFX5_IMPI_DIR=/apps/chpc/compmech/CFD/ansys_inc/v242/commonfiles/MPI/Intel/2018.3.222/linx64
+cfx=/apps/chpc/compmech/CFD/ansys_inc/v242/CFX/bin/cfx5solve #path to cfx5 solver
+$cfx -batch -def testCFX.def -par-dist $(tr "\n" "," <$PBS_NODEFILE | rev | cut -c2- | rev) > runcfx.log
+</file>
+===== Different methods of uploading a simulation =====
+==== Build and test locally, upload .def file ====
+The "standard" process assumes that the user already has a local license for the software.
+   * Mesh and pre-process the simulation as usual for a local simulation.
+   * Test it locally to ensure that everything works properly. Be cautious about absolute path file names.
+   * Compress the def file with gzip.
+   * Upload to CHPC using either scp or rsync.  The advantage of rsync is that the transfer can be made persistent, to prevent network communication glitches from killing the file transfer.
+==== Build and test locally, upload geometry and script files only, mesh and pre-process remotely ====
+If your simulations files are too large, or your internet connection too slow, consider transferring geometry and script files only.  This will require careful scripting and testing, but is certainly practical.
+   * Neither ANSYS-Mesh nor Gambit is available on the CHPC system, but ICEMCFD is.
+   * If using IcemCFD, transfer the Icem .prj, .tin, .fbc and .blk (if using hexa) files, along with a recorded Icem script file for generating the mesh and exporting the file in CFX format.  Watch for absolute path names in the script file.  Run IcemCFD with the -batch -script options to create the mesh.  A more comprehensive CFX script will be required to import the mesh and pre-process the case. **Test locally!**
+   * If your internet connection is too slow to permit easy case uploading, it will also be far too slow for downloading the results files.  Consider generating post-processing images "on the fly", or alternatively exporting only surface data on completion of the simulation.
+===== General tips and advice =====
+   * Give some thought to the resources being requested.  Partitioning a simulation too finely will not necessarily speed it up as expected.  In general, less than 100 000 cells per process is seldom justified.
+   * A request for a smaller number of cores may result in the job launching earlier, resulting in reduced turn-around time, even if the job takes longer to run.
+   * Monitoring convergence of batch jobs can be painful but necessary.
+   * Monitor files (such as cd or cl files) can be plotted with gnuplot even if no GUI is available.  On a slow connection, consider using gnuplot with ''set term dumb'' to get funky 1970's style ASCII graphics.
+   *  If you need to submit a large number of small jobs, when doing a parametric study, for example, please use Job Arrays.  Refer to the PBS-Pro guide at http://wiki.chpc.ac.za/quick:pbspro for guidance on how to set this up.

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools