wiki.chpc.ac.za

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Ansys Mechanical at the CHPC ======
The Ansys Mechanical sofware is installed together with the CFD solvers in the directory ''/apps/chpc/compmech/CFD/ansys_inc''.  There is a separate sub-directory for each version up to v221 although some of the older versions have been removed in order to save storage space.  

===== Setting up runs =====
The most efficient way of using the cluster is by means of pre-written files for the mechanical solver.  However, if you need to use Workbench to set up the runs on Lengau, it will be necessary to use one of the visualisation nodes - chpcviz1 or chpclic1.  Please refer to the [[howto:remote_viz|Remote Visualisation Instructions]] for a method of getting a graphics-enabled remote desktop on Lengau.  Workbench can be started with a command like  <code> /opt/VirtualGL/bin/vglrun /apps/chpc/compmech/CFD/ansys_inc/v195/Framework/bin/Linux64/runwb2 & </code>  Please bear in mind that these are shared servers, and computationally intensive tasks are not permitted.  In this example command we have loaded Workbench for version 195.  Please use the version that is appropriate to your purpose.

The Ansys RSM (Remote Solver Management) method has turned out to be tricky to use with the cluster scheduler, and it is far easier to use the GUI software to write input files for the mechanical solver.  This can be done by selecting the appropriate Solver section in the tree in the Mechanical Interface, and then using the menu options ''Tools - Write Input File'' to create a suitably named .dat file in a directory of your choice.  Once this has been done, the GUI and Workbench are no longer needed.  A PBS script (see below) must then be created in the appropriate directory, and it can be submitted to the cluster with the usual qsub command.  On completion of the run, the result files can be loaded into the Mechanical GUI for post-processing.

===== Example job script =====

<file bash runMechanical.pbs>
#!/bin/bash
##### The following line will request 2 (virtual) nodes, each with 24 cores running 24 mpi processes for
##### a total of 48-way parallel.  Specifying memory requirement is unlikely to be necessary, as the 
##### compute nodes have 128 GB each.
#PBS -l select=2:ncpus=24:mpiprocs=24:mem=120GB:nodetype=haswell_reg
## For your own benefit, try to estimate a realistic walltime request.  Over-estimating the 
## wallclock requirement interferes with efficient scheduling, will delay the launch of the job,
## and ties up more of your CPU-time allocation untill the job has finished.
#PBS -q normal
#PBS -P myprojectcode
#PBS -l walltime=1:00:00
#PBS -o /mnt/lustre/users/username/MechTesting/mechJob.out
#PBS -e /mnt/lustre/users/username/MechTesting/mechJob.err
#PBS -m abe
#PBS -M username@email.co.za
##### Running commands
### Tell it where to find the license
export LM_LICENSE_FILE=1055@login1
export ANSYSLMD_LICENSE_FILE=1055@login1
### You may need the Intel compiler to be available
module load chpc/parallel_studio_xe/18.0.2/2018.2.046
### Mesa is needed to provide libGLU.so
module load chpc/compmech/mesa/18.1.9
#### There is no -d option available under PBS Pro, therefore 
#### explicitly set working directory and change to that.
export PBS_JOBDIR=/mnt/lustre3p/users/username/MechTesting
cd $PBS_JOBDIR
### Count the number of lines in the machinefile
nproc=`cat $PBS_NODEFILE | wc -l`
### Get a file with only one line per host
cat $PBS_NODEFILE | sort -u > hostlist
### Create a machinefile with the number of MPI processes per node appended to each hostname
sed 's/.cm.cluster/:24/g'  hostlist > hosts
### Select the solver that you want to run
exe=/apps/chpc/compmech/CFD/ansys_inc/v190/ansys/bin/ansys190
### Command to execute the solver
$exe -b nolist -s noread -i name_of_input_file.dat -o some_output.out -np $nproc -dis -machinefile hosts 
</file>
wiki.chpc.ac.za

User Tools

Site Tools

Page Tools
