User Tools

Site Tools


howto:rocky_dem

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Last revision Both sides next revision
howto:rocky_dem [2020/02/10 10:56]
ccrosby created
howto:rocky_dem [2020/02/10 13:18]
ccrosby
Line 15: Line 15:
 #PBS -l select=1:​ncpus=20:​mpiprocs=20:​ngpus=2 -q gpu_2  #PBS -l select=1:​ncpus=20:​mpiprocs=20:​ngpus=2 -q gpu_2 
 ## Specify your own project shortcode here ## Specify your own project shortcode here
 +##
 #PBS -P RCHPC  #PBS -P RCHPC 
 ## The walltime should be a small overestimate of the expected run time ## The walltime should be a small overestimate of the expected run time
 ##  Requesting a very long walltime may delay the start of your job ##  Requesting a very long walltime may delay the start of your job
 ##  If the requested walltime is too short, the job will be killed before it is finished ##  If the requested walltime is too short, the job will be killed before it is finished
 +##
 #PBS -l walltime=2:​00:​00 #PBS -l walltime=2:​00:​00
 ##  Obviously use your own paths here ##  Obviously use your own paths here
 +##
 #PBS -e /​home/​jblogs/​lustre/​Rocky_Run/​stderr.txt #PBS -e /​home/​jblogs/​lustre/​Rocky_Run/​stderr.txt
 #PBS -o /​home/​jblogs/​lustre/​Rocky_Run/​stdout.txt #PBS -o /​home/​jblogs/​lustre/​Rocky_Run/​stdout.txt
 ## These two lines will send you an email on Abort, Begin and End of the job ## These two lines will send you an email on Abort, Begin and End of the job
 ##  Obviously use your own real email address ##  Obviously use your own real email address
 +##
 #PBS -m abe #PBS -m abe
 #PBS -M jblogs@email.co.za #PBS -M jblogs@email.co.za
 ## Tell the system where your Rocky case is ## Tell the system where your Rocky case is
 +##
 export PBS_JOBDIR=/​home/​jblogs/​lustre/​Rocky_Run export PBS_JOBDIR=/​home/​jblogs/​lustre/​Rocky_Run
 ## Change into that directory ## Change into that directory
 +##
 cd $PBS_JOBDIR cd $PBS_JOBDIR
-## There is no license available on this particular port.  Licensing needs to be negotiated with the software vendor+## There is no license available on this particular port.  Licensing needs to be negotiated with the software vendor
 +##
 export LM_LICENSE_FILE=4321@chpclic1 export LM_LICENSE_FILE=4321@chpclic1
 ## Parse the environment variable to find the GPU(s) assigned to you, and pass this information to Rocky. ## Parse the environment variable to find the GPU(s) assigned to you, and pass this information to Rocky.
 +##  Rocky does not need to know the indices of the actual GPUs, it just needs to correct number of 
 +##   ​sequential --gpu-num= ​ entries, starting at 0 and incrementing by 1.  PBS takes care of process placement.
 +## 
 IFS=', ' read -r -a gpulist <<<​ "​$CUDA_VISIBLE_DEVICES"​ IFS=', ' read -r -a gpulist <<<​ "​$CUDA_VISIBLE_DEVICES"​
 gpulistparameter=""​ gpulistparameter=""​
 +gpucount=0
 for element in "​${gpulist[@]}"​ for element in "​${gpulist[@]}"​
- do + do 
-     gpulistparameter=$gpulistparameter"​ --gpu-num="​$element +     gpulistparameter=$gpulistparameter"​ --gpu-num="​$gpucount 
- done+     let gpucount=$gpucount+1 
 + done
 echo $gpulistparameter echo $gpulistparameter
 ## Run the solver ## Run the solver
 +##
 /​apps/​chpc/​compmech/​Rocky/​rocky4/​Rocky --simulate ​ "​./​MyRockyCase.rocky"​ --resume=1 --use-gpu=1 ​ $gpulistparameter /​apps/​chpc/​compmech/​Rocky/​rocky4/​Rocky --simulate ​ "​./​MyRockyCase.rocky"​ --resume=1 --use-gpu=1 ​ $gpulistparameter
 </​file>​ </​file>​
 +
 +===== Using the GUI and Postprocessing =====
 +The V100 GPUs in the GPU cluster are not available for graphics use.  We are working on making the GUI available on the GPU cluster by means of Mesa software rendering, but this does not work yet.  Please use the dedicated visualization servers chpcviz1 or chpclic1 if you need to use the GUI or to display results. ​ Follow the [[howto:​remote_viz|remote visualization]] instructions for the TurboVNC / VirtualGL software stack.
/var/www/wiki/data/pages/howto/rocky_dem.txt · Last modified: 2020/02/10 23:31 by ccrosby