wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » howto » cfd-ace
Trace: • commercial_codes • licensing • amber • ml • globus • tune • mfix • windows • starccm
howto:cfd-ace

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision 		Previous revision
howto:cfd-ace [2015/03/16 13:56]
ccrosby created 		howto:cfd-ace [2021/12/09 16:42] (current)
Line 1: Line 1:
 ====== ESI CFD-Ace ====== ====== ESI CFD-Ace ======
  
-The CHPC has an installation of the CFD-Ace suite of software, including Ace, Fastran, Geom, View and VisCART, but with no license.  Approved academic users can use the software on the CHPC cluster, but need to work off their own licenses.  Version 2013.0 is currently available on the system.  If you need an older version, please contact CHPC support through the CHPC website.+The CHPC has an installation of the CFD-Ace suite of software, including Ace, Geom, View and VisCART, but with no license.  Approved academic users can use the software on the CHPC cluster, but need to work off their own licenses.  Version 2015.0 is currently available on the system.  If you need an older version, please contact CHPC support through the CHPC website.
  
  
 ===== Installation ===== ===== Installation =====
  
-The ESI-CFD suite is installed under '''/opt/gridware/non-supported/ESI'''. +The ESI-CFD suite is installed under '''/apps/chpc/compmech/CFD/ESI'''
  
  
 ===== Licensing ===== ===== Licensing =====
  
-Because the cluster compute nodes do not have direct access to the internet, it is necessary to use ssh-tunneling through the license server node chpcsp03 node to contact the user's license server, which will need to have the license manager and vendor daemon ports open.+Because the cluster compute nodes do not have direct access to the internet, it is necessary to use ssh-tunneling through another node to contact the user's license server, which will need to have the license manager and vendor daemon ports open.  Tunneling is done via a separate node, chpcslic1.  Please apply to  [[http://www.chpc.ac.za/index.php/support-resources/log-a-support-query|CHPC Support]] so that the CHPC firewall can be configured to allow the necessary traffic.  You will need to take similar firewall configuration measures at your end.
  
  
 ===== Running a CFD-Ace Job ===== ===== Running a CFD-Ace Job =====
  
-On the CHPC clusters all simulations are submitted as jobs to the PBS Pro job scheduler which will assign your job to the appropriate queue and machine.  CFD-Ace runs on the three Intel Xeon based clusters: Nehalem, Westmere and Dell.+On the CHPC clusters all simulations are submitted as jobs to the PBS Pro job scheduler which will assign your job to the appropriate queue and machine.  Unlike other software packages, CFD-Ace is not compatible with the parallel Lustre scratch file system.  Use your home directory instead.
  
 Example job script: Example job script:
  
-<code>+<file bash runAce.qsub>
 #!/bin/bash #!/bin/bash
-##### The following line will request 2 (virtual) nodes, each with 12 cores running 12 mpi processes  +##### The following line will request 10 (virtual) nodes, each with 24 cores running 24 mpi processes  
-##### a total of 24-way parallel. jobtype can be dell, westmere or nehalem +##### a total of 24-way parallel. 
-#PBS -l select=2:ncpus=12:mpiprocs=12:mem=4GB:jobtype=dell,place=free:group=nodetype +#PBS -l select=10:ncpus=24:mpiprocs=24:mem=4GB 
-#PBS -q workq+#PBS -q normal 
 +##### Supply YOUR resource programme code in the next line 
 +#PBS -P MECH0000
 #PBS -l walltime=1:00:00 #PBS -l walltime=1:00:00
-#PBS -o /export/home/username/scratch/acetesting/ace.out +#### CFD-Ace will not work on scratch, work in home directory: 
-#PBS -e /export/home/username/scratch/acetesting/ace.err+#PBS -o /home/username/acetesting/ace.out 
 +#PBS -e /home/username/acetesting/ace.err
 ##### The following two lines will send the user an e-mail when the job aborts, begins or ends. ##### The following two lines will send the user an e-mail when the job aborts, begins or ends.
 #PBS -m abe #PBS -m abe
Line 34: Line 37:
 ##### Set up path.  To ensure that the compute nodes also have access to this path,  ##### Set up path.  To ensure that the compute nodes also have access to this path, 
 #####  also add these lines to your .bashrc file #####  also add these lines to your .bashrc file
-export ESI_HOME=/opt/gridware/non-supported/ESI +export ESI_HOME=/apps/chpc/compmech/CFD/ESI 
-export PATH=$ESI_HOME/2013.0/UTILS/bin:$PATH +export PATH=$ESI_HOME/2015.0/UTILS/bin:$PATH 
-export LD_LIBRARY_PATH=$ESI_HOME/2013.0/UTILS/lib:$LD_LIBRARY_PATH+export LD_LIBRARY_PATH=$ESI_HOME/2015.0/UTILS/lib:$LD_LIBRARY_PATH
 ##### set up ssh-tunnels to your license server. Obviously use the right port numbers. And server URL. ##### set up ssh-tunnels to your license server. Obviously use the right port numbers. And server URL.
 #### lmgrd daemon #### lmgrd daemon
-ssh -f username@chpcsp03 -L 1999:licenseserver.ac.za:1999 -N+ssh -f username@chpclic1 -L 1999:licenseserver.ac.za:1999 -N
 #### vendor daemon port #### vendor daemon port
-ssh -f username@chpcsp03 -L 1998:licenseserver.ac.za:1998 -N+ssh -f username@chpclic1 -L 1998:licenseserver.ac.za:1998 -N
 #### Tell solver where to look for the license.   #### Tell solver where to look for the license.  
 ####  localhost is correct here, it follows from the ssh-tunneling ####  localhost is correct here, it follows from the ssh-tunneling
-export LM_LICENSE_FILE=1999localhost+export LM_LICENSE_FILE=1999@localhost
 #### There is no -d option available under PBS Pro, therefore  #### There is no -d option available under PBS Pro, therefore 
 #### explicitly set working directory and change to that. #### explicitly set working directory and change to that.
-export PBS_JOBDIR=/export/home/username/scratch/acetesting+#### CFD-Ace will not work on scratch, work in home directory. 
 +export PBS_JOBDIR=/home/username/acetesting
 cd $PBS_JOBDIR cd $PBS_JOBDIR
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
 #### This is a minimal run instruction,  #### This is a minimal run instruction, 
 ####  it will run the solver until reaching the stopping criterium set in the DTF file. ####  it will run the solver until reaching the stopping criterium set in the DTF file.
-CFD-SOLVER --model test01 --wd=$PBS_JOBDIR --num=$nproc --hosts=$PBS_NODEFILE --sim=1 > test01.out+CFD-SOLVER --model=test01 --wd=$PBS_JOBDIR --num=$nproc --hosts=$PBS_NODEFILE --sim=1 > test01.out
  
-</code>+</file>
  
 ===== Monitoring a CFD-Ace Job ===== ===== Monitoring a CFD-Ace Job =====
  
-There are different ways of monitoring your solution.  Please refer to the page on [[howto:remote_viz|Remote Visualization]] for instructions on how to get a VNC session on the visualization server.  Add the path to the VirtualGL installation with the environment setting: '''export PATH=$PATH:/opt/VirtualGL/bin'''. The vglrun wrapper is required to ensure that OpenGL works in the VNC session.  In order to access your license, set up the environment (with the bash '''export''' statements as per your job submission script), and create the same SSH-tunnels as per the job submission script.  The command '''vglrun CFD-ACE-GUI''' should allow you to open the GUI. Once you have the GUI open, you can open the .RSL and .MON files to monitor progress.  Use a similar process to open CFD-VIEW for graphical post-processing and image creation.+There are different ways of monitoring your solution.  Please refer to the page on [[howto:remote_viz|Remote Visualization]] for instructions on how to get a VNC session on the visualization server.  Add the path to the VirtualGL installation with the environment setting: '''export PATH=$PATH:/opt/VirtualGL/bin'''. The vglrun wrapper is required to ensure that OpenGL works in the VNC session. Alternatively, as per the remote viz instructions, simply start your X-session with the vglrun command, after which it will not be necessary to use it again for OpenGL apps. In order to access your license, set up the environment (with the bash '''export''' statements as per your job submission script), and create the same SSH-tunnels as per the job submission script.  The command '''vglrun CFD-ACE-GUI''' should allow you to open the GUI. Once you have the GUI open, you can open the .RSL and .MON files to monitor progress.  Alternatively, you can use gnuplot to plot the contents of these files.  In the absence of a graphics-capable environment, it is still practical to plot these files with gnuplot, using old-fashioned but very funky ASCII graphics.  In the gnuplot interface, simply give the command '''set terminal dumb 120 40''', where the numbers specify the size of the character terminal. Use vglrun to open CFD-VIEW for graphical post-processing and image creation.
  
  
/app/dokuwiki/data/attic/howto/cfd-ace.1426506960.txt.gz · Last modified: 2021/12/09 16:42 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki