wiki.chpc.ac.za

User Tools

Site Tools

You are here: start » howto » ultrafluidx
Trace: • openfoam • pythia8
howto:ultrafluidx

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
howto:ultrafluidx [2022/03/29 16:09]
ccrosby [Example PBS scripts] 		howto:ultrafluidx [2022/03/29 16:23] (current)
ccrosby [Example PBS scripts]
Line 28: Line 28:
   * To monitor the progress of the job that is writing the output file uFX_log_2022-03-28_08-27-50.out <code> tail -f uFX_log_2022-03-28_08-27-50.out </code>  To exit press CTRL-C.   * To monitor the progress of the job that is writing the output file uFX_log_2022-03-28_08-27-50.out <code> tail -f uFX_log_2022-03-28_08-27-50.out </code>  To exit press CTRL-C.
   * To view GPU usage, get the hostname of the node being used, then ssh to it.  This is only possible as long as you have a job running on it.  Then run the command <code> /apps/chpc/compmech/nvtop/bin/nvtop </code>  To exit press the ''q'' key.       * To view GPU usage, get the hostname of the node being used, then ssh to it.  This is only possible as long as you have a job running on it.  Then run the command <code> /apps/chpc/compmech/nvtop/bin/nvtop </code>  To exit press the ''q'' key.    
 +
 +
 +=== Relationship between the number of MPI ranks and the number of GPUs ===
 +UltraFluidX requires one more MPI rank than the number of GPUs.  Please refer to this table:
 +
 +^ Number of GPUs ^ ncpus ^ mpiprocs ^ ngpus ^ queue ^ Command line  ^
 +|  1  |  2  |  2  |  1  | gpu_1 | ufx -np 2 -inpFile .... |
 +|  2  |  3  |  3  |  2  | gpu_2 | ufx -np 3 -inpFile .... |
 +|  3  |  4  |  4  |  3  | gpu_3 | ufx -np 4 -inpFile .... |
 +|  4  |  5  |  5  |  4  | gpu_4 | ufx -np 5 -inpFile .... |
 +
 +
  
  
Line 65: Line 77:
 <code> qsub -l select=1:ncpus=5:mpiprocs=5:ngpus=4 -q gpu_4 -P MECH1234 -l walltime=4:00:00 </code> <code> qsub -l select=1:ncpus=5:mpiprocs=5:ngpus=4 -q gpu_4 -P MECH1234 -l walltime=4:00:00 </code>
  
-It gets more difficult if only two NVlinked GPUs are required.  It is then necessary to nominate a particular NVlink-equipped compute node:+It gets more difficult if only two or three NVlinked GPUs are required.  It is then necessary to nominate a particular NVlink-equipped compute node:
  
 <code> qsub -l select=1:ncpus=3:mpiprocs=3:ngpus=2:host=gpu4001 -q gpu_4 -P MECH1234 -l walltime=4:00:00 </code> <code> qsub -l select=1:ncpus=3:mpiprocs=3:ngpus=2:host=gpu4001 -q gpu_4 -P MECH1234 -l walltime=4:00:00 </code>
/app/dokuwiki/data/attic/howto/ultrafluidx.1648562946.txt.gz · Last modified: 2022/03/29 16:09 by ccrosby

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
CC Attribution-Noncommercial-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki