| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
howto:tune [2011/02/08 17:05]
smabakane |
howto:tune [2021/12/09 16:42] (current)
|
| ====== User Guide For Tuning and Analysis Tools in CHPC Sun System ====== | ====== User Guide For Tuning and Analysis Tools on the CHPC Lengau System ====== |
| |
| Author: M.S Mabakane | Author: M.S Mabakane |
| Date: February, 08 2011 | Date: November, 30 2018 |
| |
| TAU (Tuning and Analysis Utilities) is a portable profiling and tracing toolkit that can be used to analyse the performance of parallel models written in Fortran, C, C++ and Python. It is also used to identify key areas such as mpi processes, message passing; in general; the performance of the application that occur during the execution of the model. Many various options are also available in the TAU package that can be selected to obtain more detail information about the performance of the model. | TAU (Tuning and Analysis Utilities) is a portable profiling and tracing tool that analyse the performance of parallel models written in C, C++, Python and FORTRAN. It is further used to identify performance bottlenecks incurred during the simulation of the parallel program. TAU users can obtain performance information such as execution time of the objects (e.g. modules, routines, subroutines and functions) and relationship between different parts of the application. |
| |
| At CHPC, TAU is installed and tested in the CHPC Sun Microsystems cluster to help users improve the performance of their models. On the other hand, TAU is also anticipated to be a useful tool for developers who need to solve challenging errors of their programs. For this task, DLPOLY-2 was selected for the purpose of testing this tuning analysis system in the CHPC cluster. However, other models such as (WRF, CAM, POP and CCSM) were also tested by the TAU Developers (University of Oregon and University of Tennessee, USA) in their supercomputing systems. | At CHPC, TAU is installed and tested on the CHPC’s Lengau cluster in order to help users improve the performance of their parallel programs. On the other hand, TAU is also useful tool for developers who need to solve complex optimization problems within their parallel applications. |
| | |
| | TAU was installed using Intel Compiler 16.0.1 and Open MPI version 1.8.8. Below is the procedure on how to port, compile and run the parallel program using TAU-2.25.1: |
| |
| Now, this is the procedure to port, compile and run the code using TAU: | Now, this is the procedure to port, compile and run the code using TAU: |
| Please note: typewriter characters depicts the command, directory or files in the system. | Please note: typewriter characters depicts the command, directory or files in the system. |
| |
| - Login to the CHPC Sun system using ''-X'' mode: e.g. ''ssh -X username@sun.chpc.ac.za'' | - Login to the CHPC Lengau system using ''–X'' mode: e.g. ''ssh –X username@lengau.chpc.ac.za''. |
| - In the command line, type the command: ''module add intel''. The system will then add the intel compiler and open-mpi in the user's environment. To test the environment type: ''which ifort'' and the following path should be displayed: ''/opt/gridware/intel/Compiler/11.1/056/bin/intel64/ifort''. | - In the command line, type the command: ''module load chpc/parallel_studio_xe/16.0.1/2016.1.150.'' The system will then add the Intel compiler in the user's environment. To test the environment type: ''which ifort'' and the following path should be displayed: ''/apps/compilers/intel/parallel_studio_xe_2016/compilers_and_libraries/linux/bin/intel64/ifort''. |
| - From then, type the command: ''module add tau''. It will add the following path to tau libraries: ''/opt/gridware/performance/tau-2.19.1/x86_64/bin'' and ''TAU_MAKEFILE=/opt/gridware/performance/tau-2.19.1/x86_64/lib/Makefile.tau-mpi-pdt-trace'' in your profile. To test the TAU environment type: ''which tau_f90.sh'', it should display these message: ''/opt/gridware/performance/tau-2.19.1/x86_64/bin/tau_f90.sh''. | - Afterwards, type the command: ''module load chpc/openmpi/1.8.8/intel-16.0.1.'' to load Open MPI-1.10.2. |
| - You are now ready to port and compile the model using TAU. Note: If your model is written in Fortran language, use the library named: ''tau_f90.sh'' to compile the code. If it is written in C language use tau_cc.sh. All TAU binaries are located in: ''/opt/gridware/performance/tau-2.19.1/x86_64/bin/''. The current installation of TAU has been performed using Intel Fortran compiler. | - From then, type the command: ''module load tau/2.25.1-intel''. It will add the following path to tau libraries: ''/apps/chpc/visualization/soft/tau-2.25.1-intel-16.0.1/x86_64/bin'' and ''TAU_MAKEFILE /apps/chpc/visualization/soft/tau-2.25.1-intel-16.0.1/x86_64/lib/Makefile.tau-icpc-mpi-pdt'' in your profile. To test the TAU environment type: ''which tau_f90.sh'', it should display these message: ''/apps/chpc/visualization/soft/tau-2.25.1-intel-16.0.1/x86_64/bin/tau_f90.sh''. |
| - Procedure to port & compile DLPOLY-2.18 with TAU-2.19.1: | - You are now ready to port and compile the model using TAU. Note: If your model is written in Fortran language, use the library named: ''tau_f90.sh'' to compile the code. If it is written in C language use tau_cc.sh. All TAU binaries are located in: ''/apps/chpc/visualization/soft/tau-2.25.1-intel-16.0.1/x86_64/bin/''. The current installation of TAU has been performed using Intel Compiler 16.0.1. |
| - Untar the dlpoly-2.18 using the following command: ''untar -xzvf packagename''. For this test, the working directory is: ''/export/home/username/scratch/dlpoly-tar/'' | - Procedure to port & compile DLPOLY-2.18 using TAU-2.25.1: |
| | - Untar the dlpoly-2.18 using the following command: ''untar -xzvf packagename''. For this test, the working directory is: ''/mnt/lustre/users/username/dlpoly-tar/'' |
| - cd to ''dl_poly_2.18/srcmod''. | - cd to ''dl_poly_2.18/srcmod''. |
| - Edit ''Makefile'' by doing the following: | - Edit ''Makefile'' by doing the following: <code> |
| <code> | BINROOT = /mnt/lustre/users/username/dlpoly-tar/executables/ # Location of executable |
| BINROOT = /export/home/username/scratch/dlpoly-tar/executables/ # Location of executable | |
| CC = icc # Intel compiler | CC = icc # Intel compiler |
| EX = DLPOLY_2.18.X # Executable name | EX = DLPOLY_2.18.X # Executable name |
| SHELL=/bin/sh | SHELL=/bin/sh |
| TYPE=par | TYPE=par |
| | |
| Sun: | |
| |
| $(MAKE) LD="tau_f90.sh -o" \ # Tau mpi wrapper (e.g. In this line, users normally put mpi wrapper such as mpif90) | $(MAKE) LD="tau_f90.sh -o" \ # Tau mpi wrapper (e.g. In this line, users normally put mpi wrapper such as mpif90) |
| FC="tau_f90.sh -c" \ # Tau mpi wrapper (e.g. in this line, users normally put an mpi wrapper such as mpif90) | FC="tau_f90.sh -c" \ # Tau mpi wrapper (e.g. in this line, users normally put an mpi wrapper such as mpif90) |
| FCFLAGS="-O3 " \ | FCFLAGS="-O3 " \ |
| EX=$(EX) BINROOT=$(BINROOT) $(TYPE) | EX=$(EX) BINROOT=$(BINROOT) $(TYPE)</code> |
| </code> | - In the same directory ''dl_poly_2.18/srcmod'', compile the model using the following command: ''make''. |
| - In the same directory ''dl_poly_2.18/srcmod'', compile the model using the following command: ''make Sun''. | - From then, type command: make install. It will then generate an executable in the following path (as quoted in your Makefile): ''/mnt/lustre/users/username/dlpoly-tar/''. Note: The only important change in your model is to replace ''mpi'' with tau wrapper. e.g. (''mpif90'' should be replaced by ''tau_f90.sh''), while ''mpicc'' should be replaced by ''tau_cc.sh''). |
| - From then, type command: make install. It will then generate an executable in the following path (as quoted in your Makefile): ''/export/home/username/scratch/dlpoly-tar/executables/''. Note: The only important change in your model is to replace ''mpi'' with tau wrapper. e.g. (''mpif90'' should be replaced by ''tau_f90.sh''), while ''mpicc'' should be replaced by ''tau_cc.sh''). | |
| - Procedure to submit the job and generate TAU output: | - Procedure to submit the job and generate TAU output: |
| -Use the normal script to submit job into the system (e.g): | -Use the normal script to submit job into the system (e.g):<code> |
| <code> | ###These lines are for PBSPro |
| ###These lines are for Moab | #!/bin/bash |
| #MSUB -l nodes=2:ppn=8 | #PBS -l select=10:ncpus=24:mpiprocs=24:nodetype=haswell_reg |
| #MSUB -l partition=ALL | #PBS -P PRJT1234 |
| #MSUB -l walltime=168:00:00 | #PBS -q normal |
| #MSUB -m be | #PBS -l walltime=4:00:00 |
| #MSUB -V | #PBS -o /mnt/lustre/users/username/dlpoly/dlpoly.out |
| #MSUB -o /export/home/username/scratch/dlpoly-2/out | #PBS -e /mnt/lustre/users/username/dlpoly/dlpoly.err |
| #MSUB -e /export/home/username/scratch/dlpoly-2/err | #PBS -m abe |
| #MSUB -d /export/home/username/scratch/dlpoly-2/ | #PBS -M your.email@address |
| #MSUB -mb | ulimit -s unlimited |
| ##### Running commands | cd /apss/chpc/executables/ |
| exe=/export/home/username/scratch/dlpoly-tar/executables/DLPOLY_2.18.X | nproc=`cat $PBS_NODEFILE | wc -l` |
| nproc=`cat $PBS_NODEFILE | wc -l` | echo nproc is $nproc |
| mpirun -np $nproc $exe | cat $PBS_NODEFILE |
| | time mpirun -np $nproc DLPOLY_2.18.X |
| </code> | </code> |
| - In this case, the job was submitted from the following directory: ''/export/home/username/scratch/dlpoly-2/''. | - In this case, the job was submitted from the following directory: ''/mnt/lustre/users/username/dlpoly/''. |
| - After the completion of the job; there should be normal output of your model including many files named: tautrace and events. | - After the completion of the job; there should produce normal output of your model including many files named: tautrace and events. |
| - Procedure to visualize the output of TAU: | - Procedure to visualize the output of TAU: |
| - In the same running directory of the job ''/export/home/username/scratch/dlpoly-2/'', type: ''tau_treemerge.pl''. Note: this command will merge all the tautrace & events files to just two files named: tau.trc and tau.edf) | - In the same running directory of the job ''/mnt/lustre/users/username/dlpoly/'', type: ''tau_treemerge.pl''. Note: this command will merge all the tautrace & events files to just two files named: tau.trc and tau.edf) |
| - Again in the same directory, type: ''tau2slog2 tau.trc tau.edf -o tau.slog2''. Note: This will then converts the traces to the slog2 format. This process may take more than 40 minutes depending on the size of the model. | - Again in the same directory, type: ''tau2slog2 tau.trc tau.edf -o tau.slog2''. Note: This will then converts the traces to the slog2 format. This process may take more than 40 minutes depending on the size of the model. |
| - From then, issue the following command: ''jumpshot tau.slog2''. It will then pop-up a message which state as follows "It seems this is your first time using Jumpshot-4, a setup file will be created in your home directory with the default settings". | - From then, issue the following command: ''jumpshot tau.slog2''. It will then pop-up a message which state as follows "It seems this is your first time using Jumpshot-4, a setup file will be created in your home directory with the default settings". |
| |
| For more information about TAU, visit the following website address: http://www.cs.uoregon.edu/research/tau/docs/newguide/bk01ch01s02.html | For more information about TAU, visit the following website address: http://www.cs.uoregon.edu/research/tau/docs/newguide/bk01ch01s02.html |
| |
| |
| |
| |
