wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » howto » ansys
Trace:
howto:ansys

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
howto:ansys [2024/02/16 14:29]
ccrosby [Running a Fluent Job] 		howto:ansys [2026/07/21 09:03] (current)
ccrosby [Exporting graphics during a run]
Line 1: Line 1:
 ====== ANSYS/Fluent ====== ====== ANSYS/Fluent ======
 +
 +===== Tutorial video - multi-node post-processing =====
 +Up to v241 it is straightforward to do post-processing on a compute node, using the Mesa software-rendering library.  However, from v242 this is only possible with a GPU-equipped visualization node.  The process is illustrated in this [[https://youtu.be/MTlygZ9CpeA|tutorial video]].
  
 ===== Command line change required from v241 ===== ===== Command line change required from v241 =====
-The v241 release of Fluent appears to parse the command line in a way that corrupts the text string returned from the ''$PBS_NODEFILE'' environment variable.  As a consequence, the slave MPI processes cannot be started up.  The simple fix is to "cat" the hostfile provided by PBS into a local file and then simply to use that instead.  This will also work for older versions.  Therefore, replace+The v241 and later releases of Fluent appear to parse the command line in a way that corrupts the text string returned from the ''$PBS_NODEFILE'' environment variable.  As a consequence, the slave MPI processes cannot be started up.  The simple fix is to "cat" the hostfile provided by PBS into a local file and then simply to use that instead.  This will also work for older versions.  Therefore, replace
 <code> <code>
 fluent 3d -t$nproc -pib -mpi=intel -cnf=$PBS_NODEFILE ..... fluent 3d -t$nproc -pib -mpi=intel -cnf=$PBS_NODEFILE .....
Line 56: Line 59:
 v232 -> /home/apps/chpc/compmech/ansys_inc/v232 v232 -> /home/apps/chpc/compmech/ansys_inc/v232
 v241 -> /home/apps/chpc/compmech/ansys_inc/v241 v241 -> /home/apps/chpc/compmech/ansys_inc/v241
 +v242 -> /home/apps/chpc/compmech/ansys_inc/v242
 </code> </code>
  
Line 166: Line 170:
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
 exe=fluent exe=fluent
-$exe 3d -t$nproc -pinfiniband -ssh -cnf=$PBS_NODEFILE -gu -driver null -i journalFile.jou > run.out+cat $PBS_NODEFILE > hosts.txt 
 +$exe 3d -t$nproc -pinfiniband -ssh -cnf=hosts.txt -gu -driver null -i journalFile.jou > run.out
  
 </file> </file>
 +
 +
 +===== Exporting graphics during a run =====
 +The CHPC gratefully acknowledges Alex Brocco for the following contribution.
 +
 +For very large parallel simulations, rendering and saving scenes natively within Fluent consistently fails, regardless of graphics driver or display settings. This occurs due to memory accumulation and host node bottlenecking.
 +
 +To bypass this bottleneck, it is recommended to set up Ansys EnSight DVS (Dynamic Visualization Store) export files using the following TUI command:
 +
 +<code> /file/transient-export/ensight-dvs-surfaces ... </code>
 +
 +This method is notably more memory‑ and CPU‑efficient than forcing the solver to render and save image files. It also supports real‑time monitoring of the simulation through Ansys EnSight via a live connection (currently untested on this cluster). Once the simulation has completed, scenes and animations can be visualized and saved by loading the DVS files into Ansys EnSight.
 +
 +Alternatively, if full flow-field data is required, one can use the following TUI command:
 +
 +<code>
 +/file/transient-export/ensight-dvs-volumes...
 +</code>
 +
 +However, this option is less efficient in terms of storage and I/O performance because data from the entire 3D volume is exported.
 +
 +**Note:** The trailing arguments of the TUI commands are case-specific. Below is a structurally valid journal execution sequence example for Ansys Fluent  2024  R1:
 +
 +<code> 
 +/file/transient-export/ensight-dvs-surfaces /mnt/lustre/users/jblogs/Stall_Sim/EnSight_ShroudPres in1:shroud_3_01 strut2:shroud_2_01 rot3:shroud_01 out4:shroud:1_01 () pressure () no "Export_ShroudPres" "time-step" 5 
 +</code>
 +
 +**Note:** If the files that are exported during multiple transient simulations are to be used as a set, all simulations must be executed on the same platform and with the same number of solver processes. This ensures that the resulting DVS files are structurally compatible and can be read together without partition mismatches.
 +
 +
 +
  
 ===== Running Fluent on GPUs (experimental) ===== ===== Running Fluent on GPUs (experimental) =====
Line 216: Line 252:
 export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase
 cd $PBS_JOBDIR cd $PBS_JOBDIR
-fluent 3d  -t1 -pinfiniband -ssh -cnf=$PBS_NODEFILE -gpuapp -gpgpu=1  -g < iterate.txt > run1gpu.out+cat $PBS_NODEFILE > hosts.txt 
 +fluent 3d  -t1 -pinfiniband -ssh -cnf=hosts.txt -gpuapp -gpgpu=1  -g < iterate.txt > run1gpu.out
 </file> </file>
  
Line 239: Line 276:
 export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase export PBS_JOBDIR=/mnt/lustre/users/jblogs/FluentGPUcase
 cd $PBS_JOBDIR cd $PBS_JOBDIR
-fluent 3d  -t3 -pinfiniband -ssh -cnf=$PBS_NODEFILE -gpuapp -gpgpu=3  -g < iterate.txt > run3gpu.out+cat $PBS_NODEFILE > hosts.txt 
 +fluent 3d  -t3 -pinfiniband -ssh -cnf=hosts.txt -gpuapp -gpgpu=3  -g < iterate.txt > run3gpu.out
 </file> </file>
  
Line 305: Line 343:
 cd /mnt/lustre/users/username/FluentTest cd /mnt/lustre/users/username/FluentTest
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
-fluent 3ddp -t$nproc -pinfiniband -ssh -mpi=intel -cnf=$PBS_NODEFILE -gu -driver null -i runCommands.txt | tee fluentrun.out+cat $PBS_NODEFILE > hosts.txt 
 +fluent 3ddp -t$nproc -pinfiniband -ssh -mpi=intel -cnf=hosts.txt -gu -driver null -i runCommands.txt | tee fluentrun.out
  
 </file> </file>
Line 324: Line 363:
  
  
 +===== Running coupled fluid / structures models =====
 +Alex Brocco of UP has done a tremendous job of not only devising how to run coupled models on the cluster, but has generously agreed to share his workflow in this document {{ :howto:systems_coupling_workflow.pdf |}}.  
 +
 +Here is the example Python script {{ :howto:run_example_full.py.gz |}}. This is the example job script:
 +
 +<file bash CoupledJobScript.pbs>
 +#!/bin/bash
 +#PBS -l select=2:ncpus=24:mpiprocs=24
 +#PBS -l cfd_base=1
 +#PBS -l anshpc=44
 +#PBS -q normal
 +#PBS -P MECH1234
 +#PBS -l walltime=0:20:00
 +#PBS -o /mnt/lustre/users/jblogs/SystemCouplingTest/PBS_Out.out
 +#PBS -e /mnt/lustre/users/jblogs/SystemCouplingTest/PBS_Err.err
 +##### Running commands
 +export LM_LICENSE_FILE=1055@login1
 +export ANSYSLMD_LICENSE_FILE=1055@login1
 +export FLUENT_ARCH=lnamd64
 +### You may need the Intel compiler to be available
 +module load chpc/parallel_studio_xe/18.0.2/2018.2.046
 +### Mesa is needed to provide libGLU.so
 +module load chpc/compmech/mesa/18.1.9
 +export PBS_JOBDIR=/mnt/lustre/users/jblogs/SystemCouplingTest
 +cd $PBS_JOBDIR
 +nproc=`cat $PBS_NODEFILE | wc -l`
 +AWP_ROOT232=/home/apps/chpc/compmech/ansys_inc/v232
 +cat $PBS_NODEFILE > hosts.txt
 +$AWP_ROOT232/SystemCoupling/bin/systemcoupling --connectiontimeout=200 --logLevel=2 --ic=infiniband --cnf=hosts.txt --sycnprocs=6 --nprocs=$nproc --runScript=run.py
 +</file>
  
  
/app/dokuwiki/data/attic/howto/ansys.1708086586.txt.gz · Last modified: 2024/02/16 14:29 by ccrosby

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki