Differences

This shows you the differences between two versions of the page.

--- howto:start [2023/07/14 13:38]
ccrosby [Transferring data to DIRISA]
+++ howto:start [2026/02/24 10:42] (current)
ccrosby [Using the Intel compilers]
@@ Line 12: / Line 12: @@
-===== Transferring data to DIRISA =====
+===== If you need more memory =====
-High-performance computing is closely related to "big data".  Many HPC users both use and generate very large data sets.  The CHPC's default recommendation to users is:
+As of May 2025, 41 compute nodes have had more memory added, cannibalizing memory modules from nodes that have failed permanently.  These 24-core nodes now have 256 GB of RAM each.  You can access these nodes by including the directive ''mem=250gb'' in your resource request line.  These nodes work with the usual queues such as smp, normal or large.
-  * Feel free to use the CHPC's lustre storage as "work space".  It is large, fast and designed for parallel computing.
-  * Lustre is **not** a safe space for storing important data sets.
-  * Expedite post-processing and remove raw data.
-  * Store post-processed data somewhere safe.
-However, the CHPC recognizes that this workflow is not universal.  Many research groups own or produce very large data sets that need to be stored safely for sharing or future re-use.  This is where the [[https://www.dirisa.ac.za/|Data Intensive Research Initiative of South Africa]] comes into play.  Like the CHPC, DIRISA is also part of South Africa's [[https://www.nicis.ac.za/|National Integrated Cyber Infrastructure System]].  DIRISA operates storage systems that **are** designed for long-term safe storage of research data.
+<code>
+#PBS -l select=1:ncpus=24:mpiprocs=24:mem=250gb
+</code>
-=== DIRISA Object Stores ===
+Only use these nodes if you need more than 128 GB of RAM, but less than the 1TB available on the "fat nodes".
-DIRISA operates two object stores:
-  * [[http://irods.org|iRODS]] is an acronym for  Integrated Rule-Oriented Data System
-  * The Long Term Archive (LTA) is a tape-based system
-These storage systems can be accessed by way of DIRISA's web-based [[https://www.dirisa.ac.za/deposit-data/|Data Deposit Tool]].  However, the data deposit tool is not well suited to the requirements of a CHPC cluster user.  It is intended for smaller quantities of data that are not already stored on NICIS resources.
-== The iCommands ==
+===== Using the Intel compilers =====
-The DIRISA object stores are directly accessible from the dtn node in the CHPC cluster.  The good news is that this data transfer node mounts all the cluster file systems, which makes it possible to transfer data more efficiently to the DIRISA object stores.  The bad news is that DIRISA's storage systems differ fundamentally from the cluster's storage:
+As of December 2025, there is no license for the Intel Parallel-studio compilers.  However, all is not lost.  The "classic" compilers and MPI are installed in the directory:
-  * The DIRISA systems operate as [[https://en.wikipedia.org/wiki/Object_storage|Object Stores]]
+<code>
-  * The cluster storage is used as
+/home/apps/chpc/compmech/compilers/intel_2021.3/oneapi
+</code>
+The most recent Lengau-compatible version of the OneAPI suite is installed in:
+<code>
+/home/apps/chpc/compmech/compilers/intel/oneapi
+</code>
+=== Classic Intel compiler and MPI ===
+To use the "classic" suite, use the following settings:
+<code>
+. /home/apps/chpc/compmech/compilers/intel_2021.3/oneapi/compiler/2021.3.0/env/vars.sh
+. /home/apps/chpc/compmech/compilers/intel_2021.3/oneapi/mkl/2021.3.0/env/vars.sh
+. /home/apps/chpc/compmech/compilers/intel_2021.3/oneapi/mpi/2021.3.0/env/vars.sh
+export CC=icc
+export CXX=icpc
+export FC=ifort
+export F90=ifort
+export F77=ifort
+export MPICC=mpiicc
+export MPICXX=mpiicpc
+export MPIFORT=mpiifort
+export MPIF77=mpiifort
+export MPIF90=mpiifort
+</code>
+It is not necessary to use a machinefile for multi-node MPI runs.
+=== OneAPI compilers and MPI ===
+Post-2024 OneAPI compilers are not compatible with Lengau's operating system.  To use the most recent compatible version, use the following settings:
+<code>
+. /home/apps/chpc/compmech/compilers/intel/oneapi/setvars.sh
+export CC=icx
+export CXX=icpx
+export FC=ifx
+export F90=ifx
+export F77=ifx
+export MPICC=mpiicx
+export MPICXX=mpiicpx
+export MPIFORT=mpiifx
+export MPIF77=mpiifx
+export MPIF90=mpiifx
+export I_MPI_HYDRA_BOOTSTRAP=ssh
+</code>
+It is not necessary to use a machinefile for multi-node MPI runs.  However, the default bootstrap method using pbsdsh does not work, hence the need to set ''I_MPI_HYDRA_BOOTSTRAP''.  This is a typical command mpi run command:
+<code>
+mpirun -np 96 myMPIprogram.exe
+</code>
+===== Transferring data to DIRISA =====
+Long-term data storage through [[howto:Dirisa|DIRISA.]]
@@ Line 56: / Line 106: @@
 ===== Dealing with software licensing =====
 Commercial software involves dealing with [[howto:licensing|licensing.]]
+===== Ssh-tunneling with MobaXterm =====
+Here is a [[https://youtu.be/lil4eagB8oY|tutorial video]] on ssh-tunneling with MobaXterm.
@@ Line 67: / Line 121: @@
   *[[ansysmechanical|Using Ansys/Mechanical at the CHPC]]
   *[[cfx|Using Ansys/CFX at the CHPC]]
+  *[[forte|Using Ansys Forte at the CHPC]]
   *[[starccm|Using STAR-CCM+ at the CHPC]]
   *[[CFD-Ace|Using ESI CFD-Ace at the CHPC]]
@@ Line 77: / Line 132: @@
   *[[howto:ultrafluidx|Using Altair's UltraFluidX LBM code at the CHPC]]
   *[[Palabos|Using the lattice-Boltzman code Palabos at the CHPC]]
+  *[[AMR-Wind|Using the wind-energy code AMR-Wind at the CHPC]]
 ===== Computational Electromagnetics =====
@@ Line 147: / Line 203: @@
   *[[p4vasp|Using p4vasp]]
   *[[vesta|Using Vesta]]
+===== Matlab at CHPC =====
+  *[[matlab|Using Matlab on the cluster]]
 ===== Earth Sciences =====
@@ Line 153: / Line 213: @@
   *[[CDO|Using Climate Data Operator (CDO)]]
+===== Windows applications =====
+  *[[windows|Running Windows applications]]

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools