wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » howto » starccm
Trace: • commercial_codes • licensing • amber • ml • globus • tune • mfix • windows • starccm
howto:starccm

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
howto:starccm [2024/11/14 10:43]
ccrosby 		howto:starccm [2026/07/13 14:11] (current)
ccrosby Added section advising users to not use dtn for license tunnels
Line 1: Line 1:
 ====== STAR-CCM+ ====== ====== STAR-CCM+ ======
  
 +===== NOTE on license tunnels =====
 +Use ''1999@chpclic1'' for the license, the dtn node is not currently available for this.
  
 ==== New procedure for versions from 19.02.009 onwards ==== ==== New procedure for versions from 19.02.009 onwards ====
Line 14: Line 16:
 </code> respectively.   </code> respectively.  
  
-Similarly, other recent versions are also in singularity image files:+Similarly, other recent versions are also in singularity images.  Siemens has been adopting different strategies of numbering the versions recently, so some of these are symbolic links:
  
 <code> <code>
Line 23: Line 25:
 /home/apps/chpc/compmech/Siemens/star19.06.008.sif /home/apps/chpc/compmech/Siemens/star19.06.008.sif
 /home/apps/chpc/compmech/Siemens/star19.06.008_r8.sif /home/apps/chpc/compmech/Siemens/star19.06.008_r8.sif
 +/home/apps/chpc/compmech/Siemens/star20.02.007.sif
 +/home/apps/chpc/compmech/Siemens/star20.02.007_r8.sif
 +/home/apps/chpc/compmech/Siemens/star20.04.007.sif
 +/home/apps/chpc/compmech/Siemens/star20.04.007_r8.sif
 +/home/apps/chpc/compmech/Siemens/starccm+_20.06.007.sif
 +/home/apps/chpc/compmech/Siemens/starccm+_20.06.007_r8.sif
 +/home/apps/chpc/compmech/Siemens/starccm+_21.02.007.sif
 +/home/apps/chpc/compmech/Siemens/starccm+_21.02.007_r8.sif
 </code> </code>
  
 +These are the two most recent singularity image files:
 +
 +<code>
 +/home/apps/chpc/compmech/Siemens/starccm+_21.02.007.sif
 +/home/apps/chpc/compmech/Siemens/starccm+_21.02.007_r8.sif
 +</code>
 +
 +Please list the contents of the directory ''/home/apps/chpc/compmech/Siemens'' to see what versions are installed and what the files are called.  Siemens is in the process of transitioning to a new version-numbering scheme, which is why we have symbolic links to offer the image files with more consistent names.  The latest version is ''2502.007'', but it is also referred to as ''20.02.007''.  Go figure.  
  
  
 To enable the Singularity environment on all the compute nodes used in a run, it is necessary to insert the following statement in your .bashrc file: To enable the Singularity environment on all the compute nodes used in a run, it is necessary to insert the following statement in your .bashrc file:
 <code> <code>
-module load chpc/singularity/3.5.3+export PATH=/usr/local/apps/singularity/3.5.3/bin:$PATH
 </code> </code>
  
Line 46: Line 64:
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
 cat $PBS_NODEFILE > hosts.txt cat $PBS_NODEFILE > hosts.txt
-singularity exec /home/apps/chpc/compmech/Siemens/star19_04_007.sif starccm+  -licpath 1999@dtn -batch run -mpi intel -power -podkey <use own 22-char PoDkey> -np $nproc -machinefile hosts.txt testCase.sim > run.out+singularity exec /home/apps/chpc/compmech/Siemens/starccm+_21.02.007.sif starccm+  -licpath 1999@chpclic1 -batch run -rsh ssh -mpi intel -power -podkey <use own 22-char PoDkey> -np $nproc -machinefile hosts.txt testCase.sim > run.out
  
 </file> </file>
Line 80: Line 98:
 ===== Licensing ===== ===== Licensing =====
  
-Because the cluster compute nodes do not have direct access to the internet, it is necessary to use ssh-tunneling through another node to contact the user's license server, which will need to have the license manager and vendor daemon ports open.  Tunneling is done via a separate node, either chpclic1 or dtn. Please apply to  [[http://users.chpc.ac.za/helpdesk|CHPC Support]] in order to supply the public IP address and port numbers of your license server so that the CHPC firewall can be configured to allow the necessary traffic.  You will need to take similar firewall configuration measures at your end.  Also please add the lines at the end of the script top kill your ssh-tunnels on completion of the job. +Because the cluster compute nodes do not have direct access to the internet, it is necessary to use ssh-tunneling through another node to contact the user's license server, which will need to have the license manager and vendor daemon ports open.  Tunneling is done via a separate node, either chpclic1 or dtn (NOTE: Ask CHPC staff for assistance, things have become more complicated as of July 2026. Please apply to  [[http://users.chpc.ac.za/helpdesk|CHPC Support]] in order to supply the public IP address and port numbers of your license server so that the CHPC firewall can be configured to allow the necessary traffic.  You will need to take similar firewall configuration measures at your end.  Also please add the lines at the end of the script top kill your ssh-tunnels on completion of the job. 
  
-For the majority of users simply checking out PoD licenses from the international STAR-CCM+ license servers, it is easiest to use the CHPC's permanent tunnels on either dtn or chpclic1 (soon to be retired).  As per the examples  scripts given below, you can simply set the license variable to point to ''1999@dtn'' or ''1999@chpclic1''.+For the majority of users simply checking out PoD licenses from the international STAR-CCM+ license servers, it is easiest to use the CHPC's permanent tunnels on chpclic1.  As per the examples  scripts given below, you can simply set the license variable to point to ''1999@chpclic1''.
  
  
Line 110: Line 128:
 export PATH=/apps/chpc/compmech/CFD/Siemens/18.06.006/STAR-CCM+18.06.006/star/bin:$PATH export PATH=/apps/chpc/compmech/CFD/Siemens/18.06.006/STAR-CCM+18.06.006/star/bin:$PATH
 ####  Tell solver where to look for the license.   ####  Tell solver where to look for the license.  
-####  dtn is correct here, there are ssh tunnels from dtn to the Siemens license servers.  +####  chpclic1 is correct here, there are ssh tunnels from chpclic1 to the Siemens license servers.  
 ####  We are following a belts, braces and modest  ####  We are following a belts, braces and modest 
 ####  underwear approach here by specifying the LM and CDLMD license files as well as giving a license path on the  ####  underwear approach here by specifying the LM and CDLMD license files as well as giving a license path on the 
 ####  command line. ####  command line.
-export LM_LICENSE_FILE=1999@dtn +export LM_LICENSE_FILE=1999@chpclic1 
-export CDLMD_LICENSE_FILE=1999@dtn+export CDLMD_LICENSE_FILE=1999@chpclic1
 #### There is no -d option available under PBS Pro, therefore  #### There is no -d option available under PBS Pro, therefore 
 #### explicitly set working directory and change to that. #### explicitly set working directory and change to that.
Line 123: Line 141:
 #### This is a minimal run instruction,  #### This is a minimal run instruction, 
 ####  it will run the solver until reaching the stopping criteria set in the sim file. ####  it will run the solver until reaching the stopping criteria set in the sim file.
-starccm+ -licpath 1999@dtn -batch run -power -podkey [your 22-character podkey] -rsh ssh -np $nproc -machinefile $PBS_NODEFILE simulationfilename.sim > run.out+starccm+ -licpath 1999@chpclic1 -batch run -power -podkey [your 22-character podkey] -rsh ssh -np $nproc -machinefile $PBS_NODEFILE simulationfilename.sim > run.out
  
 </file> </file>
Line 165: Line 183:
 #PBS -o /mnt/lustre/users/jblogs/starccmGPUcase/star_1gpu.out #PBS -o /mnt/lustre/users/jblogs/starccmGPUcase/star_1gpu.out
 #PBS -e /mnt/lustre/users/jblogs/starccmGPUcase/star_1gpu.err #PBS -e /mnt/lustre/users/jblogs/starccmGPUcase/star_1gpu.err
-export LM_LICENSE_FILE=1999@dtn +export LM_LICENSE_FILE=1999@chpclic1 
-export CDLMD_LICENSE_FILE=1999@dtn+export CDLMD_LICENSE_FILE=1999@chpclic1
 # Edit this next line to select the appropriate version. # Edit this next line to select the appropriate version.
 export PATH=/home/apps/chpc/compmech/Siemens/17.06.007/STAR-CCM+17.06.007/star/bin:$PATH export PATH=/home/apps/chpc/compmech/Siemens/17.06.007/STAR-CCM+17.06.007/star/bin:$PATH
Line 172: Line 190:
 export PBS_JOBDIR=/mnt/lustre/users/jblogs/starccmGPUcase export PBS_JOBDIR=/mnt/lustre/users/jblogs/starccmGPUcase
 cd $PBS_JOBDIR cd $PBS_JOBDIR
-starccm+ -licpath 1999@dtn -batch run -power -podkey 12345ABCDE09876fghijk0 -rsh ssh -np $nproc -machinefile $PBS_NODEFILE -gpgpu auto:1 mysimfile.sim > run.out+starccm+ -licpath 1999@chpclic1 -batch run -power -podkey 12345ABCDE09876fghijk0 -rsh ssh -np $nproc -machinefile $PBS_NODEFILE -gpgpu auto:1 mysimfile.sim > run.out
 </file> </file>
  
Line 185: Line 203:
 #PBS -o /mnt/lustre/users/jblogs/starccmGPUcase/star_3gpu.out #PBS -o /mnt/lustre/users/jblogs/starccmGPUcase/star_3gpu.out
 #PBS -e /mnt/lustre/users/jblogs/starccmGPUcase/star_3gpu.err #PBS -e /mnt/lustre/users/jblogs/starccmGPUcase/star_3gpu.err
-export LM_LICENSE_FILE=1999@dtn +export LM_LICENSE_FILE=1999@chpclic1 
-export CDLMD_LICENSE_FILE=1999@dtn+export CDLMD_LICENSE_FILE=1999@chpclic1
 # Edit this next line to select the appropriate version. # Edit this next line to select the appropriate version.
 export PATH=/home/apps/chpc/compmech/Siemens/17.06.007/STAR-CCM+17.06.007/star/bin:$PATH export PATH=/home/apps/chpc/compmech/Siemens/17.06.007/STAR-CCM+17.06.007/star/bin:$PATH
Line 192: Line 210:
 export PBS_JOBDIR=/mnt/lustre/users/jblogs/starccmGPUcase export PBS_JOBDIR=/mnt/lustre/users/jblogs/starccmGPUcase
 cd $PBS_JOBDIR cd $PBS_JOBDIR
-starccm+ -licpath 1999@dtn -batch run -power -podkey 12345ABCDE09876fghijk0 -rsh ssh -np $nproc -machinefile $PBS_NODEFILE -gpgpu auto:3 mysimfile.sim > run.out+starccm+ -licpath 1999@chpclic1 -batch run -power -podkey 12345ABCDE09876fghijk0 -rsh ssh -np $nproc -machinefile $PBS_NODEFILE -gpgpu auto:3 mysimfile.sim > run.out
 </file> </file>
  
Line 221: Line 239:
 #### Get the total number of cores #### Get the total number of cores
 nproc=`cat $PBS_NODEFILE | wc -l` nproc=`cat $PBS_NODEFILE | wc -l`
-# Build SSH tunnels for license servers.   +# Build SSH tunnels for DM license server.   
-#  You may also prefer to use the existing tunnels on chpclic1 or dtn, see the simpler example above+# Use the existing tunnels on chpclic1 or chpclic1 for the PoD license
 # Assume that DM license is in port 27120 on the server license.unseen.ac.za  # Assume that DM license is in port 27120 on the server license.unseen.ac.za 
 # STAR-CCM+ will look first for a PoD license # STAR-CCM+ will look first for a PoD license
Line 228: Line 246:
 #  consume PoD time very quickly #  consume PoD time very quickly
 masternode=`hostname -s` masternode=`hostname -s`
-ssh -f jblogs@chpclic1 -L *:1999:flex.cd-adapco.com:1999 -N 
-ssh -f jblogs@chpclic1 -L *:2099:flex.cd-adapco.com:2099 -N 
 ssh -f jblogs@chpclic1 -L *:27120:license.unseen.ac.za:27120 -N ssh -f jblogs@chpclic1 -L *:27120:license.unseen.ac.za:27120 -N
 ssh -f jblogs@chpclic1 -L *:27121:license.unseen.ac.za:27121 -N ssh -f jblogs@chpclic1 -L *:27121:license.unseen.ac.za:27121 -N
-export CDLMD_LICENSE_FILE=1999@$masternode:27120@$masternode +export CDLMD_LICENSE_FILE=1999@chpclic1:27120@$masternode 
-export LM_LICENSE_FILE=1999@$masternode:27120@$masternode+export LM_LICENSE_FILE=1999@chpclic1:27120@$masternode
 ##### Starccm+ run instruction. ##### Starccm+ run instruction.
 starlaunch jobmanager  --command "$STARCCMHOME/star/bin/starccm+ -licpath 1999@$masternode:27120@$masternode -power -podkey <22-characterPoDKey>  -rsh ssh -batch run -mpi intel -np $nproc -machinefile $PBS_NODEFILE  ExampleDM_Project.dmprj" --slots 0  --batchsystem pbs  >> disasstar.out 2>&1 starlaunch jobmanager  --command "$STARCCMHOME/star/bin/starccm+ -licpath 1999@$masternode:27120@$masternode -power -podkey <22-characterPoDKey>  -rsh ssh -batch run -mpi intel -np $nproc -machinefile $PBS_NODEFILE  ExampleDM_Project.dmprj" --slots 0  --batchsystem pbs  >> disasstar.out 2>&1
 ### Remove the ssh tunnels: ### Remove the ssh tunnels:
-kill -9 `ps ux | grep "ssh -f" |  grep -o -E '[0-9]+' | head -1 | sed -e 's/^0\+//' ` 
-kill -9 `ps ux | grep "ssh -f" |  grep -o -E '[0-9]+' | head -1 | sed -e 's/^0\+//' ` 
 kill -9 `ps ux | grep "ssh -f" |  grep -o -E '[0-9]+' | head -1 | sed -e 's/^0\+//' ` kill -9 `ps ux | grep "ssh -f" |  grep -o -E '[0-9]+' | head -1 | sed -e 's/^0\+//' `
 kill -9 `ps ux | grep "ssh -f" |  grep -o -E '[0-9]+' | head -1 | sed -e 's/^0\+//' ` kill -9 `ps ux | grep "ssh -f" |  grep -o -E '[0-9]+' | head -1 | sed -e 's/^0\+//' `
Line 293: Line 307:
 </code> </code>
  
-However, because there are a lot of users on Lengau using STAR-CCM+ this way, we have set up permanent ssh tunnels through both chpclic1 and dtn.  That means that the power on demand license appears to be available on chpclic1 and dtn, so there is no need for your own tunnels.+However, because there are a lot of users on Lengau using STAR-CCM+ this way, we have set up permanent ssh tunnels through chpclic1.  That means that the power on demand license appears to be available on chpclic1, so there is no need for your own tunnels.
  
 <code> <code>
Line 300: Line 314:
 </code> </code>
  
-The server chpclic1 will shortly be retired, therefore we recommend that you use dtn instead.+The server chpclic1 will shortly be retired, therefore we recommend that you use chpclic1 instead.
  
 <code> <code>
-export LM_LICENSE_FILE=1999@dtn +export LM_LICENSE_FILE=1999@chpclic1 
-export CDLMD_LICENSE_FILE=1999@dtn+export CDLMD_LICENSE_FILE=1999@chpclic1
 </code> </code>
  
/app/dokuwiki/data/attic/howto/starccm.1731573807.txt.gz · Last modified: 2024/11/14 10:43 by ccrosby

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki