wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » howto » ansys
Trace:
howto:ansys

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
howto:ansys [2023/08/01 15:39]
ccrosby [Installation] 		howto:ansys [2026/07/21 09:03] (current)
ccrosby [Exporting graphics during a run]
Line 1: Line 1:
 ====== ANSYS/Fluent ====== ====== ANSYS/Fluent ======
  
-===== LICENSING PROBLEMS =====+===== Tutorial video - multi-node post-processing ===== 
 +Up to v241 it is straightforward to do post-processing on a compute node, using the Mesa software-rendering library.  However, from v242 this is only possible with a GPU-equipped visualization node.  The process is illustrated in this [[https://youtu.be/MTlygZ9CpeA|tutorial video]].
  
-**New problem, as of 11 AM on 25 March 2023.  It is not possible to start the Ansys license server on login1 due to a library compatibility problem following the emergency shutdown.  For now, the only available Ansys license is on chpclic1, and this works only for pre-R21.1 versions of the software.**+===== Command line change required from v241 ===== 
 +The v241 and later releases of Fluent appear to parse the command line in a way that corrupts the text string returned from the ''$PBS_NODEFILE'' environment variable.  As a consequence, the slave MPI processes cannot be started up.  The simple fix is to "cat" the hostfile provided by PBS into a local file and then simply to use that instead.  This will also work for older versions.  Therefore, replace 
 +<code> 
 +fluent 3d -t$nproc -pib -mpi=intel -cnf=$PBS_NODEFILE ..... 
 +</code>  
 +with 
 +<code> 
 +cat $PBS_NODEFILE > hosts.txt 
 +fluent 3d -t$nproc -pib -mpi=intel -cnf=hosts.txt ..... 
 +</code>  
 + 
 + 
 +===== LICENSING PROBLEMS =====
  
 Please bear with us while we work with Ansys on sorting out some teething problems introduced by the license system changes.    Please bear with us while we work with Ansys on sorting out some teething problems introduced by the license system changes.   
Line 45: Line 58:
 v231 -> /home/apps/chpc/compmech/ansys_inc/v231 v231 -> /home/apps/chpc/compmech/ansys_inc/v231
 v232 -> /home/apps/chpc/compmech/ansys_inc/v232 v232 -> /home/apps/chpc/compmech/ansys_inc/v232
 +v241 -> /home/apps/chpc/compmech/ansys_inc/v241
 +v242 -> /home/apps/chpc/compmech/ansys_inc/v242
 </code> </code>
  
Line 110: Line 125:
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
 exe=fluent exe=fluent
-$exe 3d -t$nproc -pinfiniband -ssh -cnf=$PBS_NODEFILE -g < fileContainingTUIcommands > run.out+cat $PBS_NODEFILE > hosts.txt 
 +$exe 3d -t$nproc -pinfiniband -ssh -cnf=hosts.txt -g < fileContainingTUIcommands > run.out
  
 </file> </file>
Line 154: Line 170:
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
 exe=fluent exe=fluent
-$exe 3d -t$nproc -pinfiniband -ssh -cnf=$PBS_NODEFILE -gu -driver null -i journalFile.jou > run.out+cat $PBS_NODEFILE > hosts.txt 
 +$exe 3d -t$nproc -pinfiniband -ssh -cnf=hosts.txt -gu -driver null -i journalFile.jou > run.out
  
 </file> </file>
 +
 +
 +===== Exporting graphics during a run =====
 +The CHPC gratefully acknowledges Alex Brocco for the following contribution.
 +
 +For very large parallel simulations, rendering and saving scenes natively within Fluent consistently fails, regardless of graphics driver or display settings. This occurs due to memory accumulation and host node bottlenecking.
 +
 +To bypass this bottleneck, it is recommended to set up Ansys EnSight DVS (Dynamic Visualization Store) export files using the following TUI command:
 +
 +<code> /file/transient-export/ensight-dvs-surfaces ... </code>
 +
 +This method is notably more memory‑ and CPU‑efficient than forcing the solver to render and save image files. It also supports real‑time monitoring of the simulation through Ansys EnSight via a live connection (currently untested on this cluster). Once the simulation has completed, scenes and animations can be visualized and saved by loading the DVS files into Ansys EnSight.
 +
 +Alternatively, if full flow-field data is required, one can use the following TUI command:
 +
 +<code>
 +/file/transient-export/ensight-dvs-volumes...
 +</code>
 +
 +However, this option is less efficient in terms of storage and I/O performance because data from the entire 3D volume is exported.
 +
 +**Note:** The trailing arguments of the TUI commands are case-specific. Below is a structurally valid journal execution sequence example for Ansys Fluent  2024  R1:
 +
 +<code> 
 +/file/transient-export/ensight-dvs-surfaces /mnt/lustre/users/jblogs/Stall_Sim/EnSight_ShroudPres in1:shroud_3_01 strut2:shroud_2_01 rot3:shroud_01 out4:shroud:1_01 () pressure () no "Export_ShroudPres" "time-step" 5 
 +</code>
 +
 +**Note:** If the files that are exported during multiple transient simulations are to be used as a set, all simulations must be executed on the same platform and with the same number of solver processes. This ensures that the resulting DVS files are structurally compatible and can be read together without partition mismatches.
 +
 +
 +
  
 ===== Running Fluent on GPUs (experimental) ===== ===== Running Fluent on GPUs (experimental) =====
Line 204: Line 252:
 export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase
 cd $PBS_JOBDIR cd $PBS_JOBDIR
-fluent 3d  -t1 -pinfiniband -ssh -cnf=$PBS_NODEFILE -gpuapp -gpgpu=1  -g < iterate.txt > run1gpu.out+cat $PBS_NODEFILE > hosts.txt 
 +fluent 3d  -t1 -pinfiniband -ssh -cnf=hosts.txt -gpuapp -gpgpu=1  -g < iterate.txt > run1gpu.out
 </file> </file>
  
Line 227: Line 276:
 export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase
 cd $PBS_JOBDIR cd $PBS_JOBDIR
-fluent 3d  -t3 -pinfiniband -ssh -cnf=$PBS_NODEFILE -gpuapp -gpgpu=3  -g < iterate.txt > run3gpu.out+cat $PBS_NODEFILE > hosts.txt 
 +fluent 3d  -t3 -pinfiniband -ssh -cnf=hosts.txt -gpuapp -gpgpu=3  -g < iterate.txt > run3gpu.out
 </file> </file>
  
Line 293: Line 343:
 cd /mnt/lustre/users/username/FluentTest cd /mnt/lustre/users/username/FluentTest
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
-fluent 3ddp -t$nproc -pinfiniband -ssh -mpi=intel -cnf=$PBS_NODEFILE -gu -driver null -i runCommands.txt | tee fluentrun.out+cat $PBS_NODEFILE > hosts.txt 
 +fluent 3ddp -t$nproc -pinfiniband -ssh -mpi=intel -cnf=hosts.txt -gu -driver null -i runCommands.txt | tee fluentrun.out
  
 </file> </file>
Line 312: Line 363:
  
  
 +===== Running coupled fluid / structures models =====
 +Alex Brocco of UP has done a tremendous job of not only devising how to run coupled models on the cluster, but has generously agreed to share his workflow in this document {{ :howto:systems_coupling_workflow.pdf |}}.  
 +
 +Here is the example Python script {{ :howto:run_example_full.py.gz |}}. This is the example job script:
 +
 +<file bash CoupledJobScript.pbs>
 +#!/bin/bash
 +#PBS -l select=2:ncpus=24:mpiprocs=24
 +#PBS -l cfd_base=1
 +#PBS -l anshpc=44
 +#PBS -q normal
 +#PBS -P MECH1234
 +#PBS -l walltime=0:20:00
 +#PBS -o /mnt/lustre/users/jblogs/SystemCouplingTest/PBS_Out.out
 +#PBS -e /mnt/lustre/users/jblogs/SystemCouplingTest/PBS_Err.err
 +##### Running commands
 +export LM_LICENSE_FILE=1055@login1
 +export ANSYSLMD_LICENSE_FILE=1055@login1
 +export FLUENT_ARCH=lnamd64
 +### You may need the Intel compiler to be available
 +module load chpc/parallel_studio_xe/18.0.2/2018.2.046
 +### Mesa is needed to provide libGLU.so
 +module load chpc/compmech/mesa/18.1.9
 +export PBS_JOBDIR=/mnt/lustre/users/jblogs/SystemCouplingTest
 +cd $PBS_JOBDIR
 +nproc=`cat $PBS_NODEFILE | wc -l`
 +AWP_ROOT232=/home/apps/chpc/compmech/ansys_inc/v232
 +cat $PBS_NODEFILE > hosts.txt
 +$AWP_ROOT232/SystemCoupling/bin/systemcoupling --connectiontimeout=200 --logLevel=2 --ic=infiniband --cnf=hosts.txt --sycnprocs=6 --nprocs=$nproc --runScript=run.py
 +</file>
  
  
/app/dokuwiki/data/attic/howto/ansys.1690897166.txt.gz · Last modified: 2023/08/01 15:39 by ccrosby

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki