quick:start
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
quick:start [2021/11/23 11:36] ccrosby [New 3 PB Lustre storage system] |
quick:start [2026/06/11 11:17] (current) kevin [Logging in for the First Time] |
||
|---|---|---|---|
| Line 6: | Line 6: | ||
| - | ===== New 3 PB Lustre storage system ===== | + | ==== Video tutorials for newcomers |
| + | If you are a new user to the CHPC, please watch these videos: | ||
| + | * [[https:// | ||
| + | * [[https:// | ||
| + | * [[https:// | ||
| + | * [[https:// | ||
| - | With effect from 1 May 2021, a new 3 PB Lustre storage system was implemented. | + | ====Mailing List==== |
| - | Please read: **[[howto:datamigration|Moving to the new 3 PB Lustre storage]]** | + | The **[[https:// |
| - | #:!: The old 4 PB Lustre storage system **will be // | ||
| - | |||
| - | ====New directory path:==== | ||
| - | |||
| - | The 3 PB Lustre is mounted on ''/ | ||
| - | |||
| - | Search and replace your script files and change ''/ | ||
| - | |||
| - | We will be updating all examples in this wiki to the new path, however, you may still come across an old example with the outmoded path. To make the example wor | ||
| =====Overview: | =====Overview: | ||
| - | The CHPC' | + | The CHPC's Dell Linux cluster |
| - | The new system is an homogeneous cluster, comprising Intel 5th generation CPUs. As of March 2017 it has 1368 compute nodes with 24 cores and 128 GiB* memory | + | The new system is an homogeneous cluster, comprising Intel 5th generation CPUs. As of May 2025 it has 41 nodes with 24 cores and 256 GiB memory, 784 compute nodes with 24 cores and 128 GiB* memory, 355 nodes with 24 cores and 64 GiB memory, and four large memory " |
| - | * Maximum available memory on each type of node: '' | + | * Maximum available memory on each type of node: '' |
| Line 42: | Line 38: | ||
| </ | </ | ||
| + | ====Logging in for the First Time==== | ||
| - | The new system is running CentOS 7.0 and uses the //Bash// shell by default. | + | :!: You will use a password only for the **first time** you log in using ssh. And the first step will be to set up your ssh key. After this, you **can only log in using ssh keys and your password will be disabled.** |
| - | You should change your password after logging in the first time. | + | Please read carefully and follow |
| - | To change | + | |
| - | Rules are: 10 characters, with at least one of the following character types: upper and lower case, numbers, and special characters. | + | |
| - | Once you have logged in, give some consideration to how you will be using your session on the login node. If you are going to spend a long time logged in, doing a variety of tasks, it is best to get yourself [[http:// | + | |
| + | ====Long Interactive Sessions==== | ||
| + | |||
| + | Once you have logged in, give some consideration to how you will be using your session on the login node. If you are going to spend a long time logged in, doing a variety of tasks, it is best to get yourself | ||
| + | |||
| + | The new system is running [[quick: | ||
| ====Trouble Logging in?==== | ====Trouble Logging in?==== | ||
| Line 79: | Line 79: | ||
| From the command line on your Linux workstation: | From the command line on your Linux workstation: | ||
| < | < | ||
| - | scp filetocopy.tar.gz yourusername@scp.chpc.ac.za:/ | + | scp filetocopy.tar.gz yourusername@scp.chpc.ac.za:/ |
| </ | </ | ||
| transfers the file // | transfers the file // | ||
| the Lustre file system on the CHPC cluster, under the // | the Lustre file system on the CHPC cluster, under the // | ||
| - | subdirectory of your scratch directory ///mnt/lustre3p/ | + | subdirectory of your scratch directory ///mnt/lustre/ |
| (where // | (where // | ||
| Line 110: | Line 110: | ||
| ^ Mount point ^ File System ^ Size ^ Quota ^ Backup ^ Access | ^ Mount point ^ File System ^ Size ^ Quota ^ Backup ^ Access | ||
| - | | ''/ | + | | ''/ |
| - | | ''/ | + | | ''/ |
| - | | ''/ | + | | ''/ |
| - | | ''/ | + | | ''/ |
| + | |||
| + | :!: **IMPORTANT NOTE:** Files older than 90 days on ''/ | ||
| - | **Note 1:** Unfortunately, | + | **Note |
| - | :!: **IMPORTANT NOTE:** Files older than 90 days on ''/ | + | **Note [2]** Create a support ticket |
| - | **Note | + | **Note |
| It is essential that all files that your job script writes to be on Lustre, apart from scheduler errors you will lose performance because your home directory is on NFS which is not a parallel file system. It is also recommended that all files your jobs scripts reads, especially if large or read more than once, be on Lustre for the same reason. | It is essential that all files that your job script writes to be on Lustre, apart from scheduler errors you will lose performance because your home directory is on NFS which is not a parallel file system. It is also recommended that all files your jobs scripts reads, especially if large or read more than once, be on Lustre for the same reason. | ||
| Line 126: | Line 128: | ||
| ====Quotas==== | ====Quotas==== | ||
| - | The ''/ | + | The ''/ |
| You can see how much you are currently using with the '' | You can see how much you are currently using with the '' | ||
| Line 137: | Line 139: | ||
| Make sure that all jobs use a working directory on the Lustre file system. | Make sure that all jobs use a working directory on the Lustre file system. | ||
| < | < | ||
| - | /mnt/lustre3p/ | + | /mnt/lustre/ |
| </ | </ | ||
| where '' | where '' | ||
| Line 256: | Line 258: | ||
| | smp | 24 | 24 | 20 | 10 | 96 | For single-node parallel jobs. | | | | smp | 24 | 24 | 20 | 10 | 96 | For single-node parallel jobs. | | | ||
| ^ normal | ^ normal | ||
| - | | large | 2400 | 264 | 10 | 5 | | + | | large | 2400 | 264 | 10 | 5 | |
| + | | xlarge | ||
| | express | | express | ||
| | bigmem | | bigmem | ||
| Line 351: | Line 354: | ||
| <code bash> | <code bash> | ||
| #!/bin/bash | #!/bin/bash | ||
| - | #PBS -l ncpus=24: | + | #PBS -l select=1:ncpus=24: |
| #PBS -P PRJT1234 | #PBS -P PRJT1234 | ||
| #PBS -q smp | #PBS -q smp | ||
| #PBS -l walltime=4: | #PBS -l walltime=4: | ||
| - | #PBS -o /mnt/lustre3p/ | + | #PBS -o /mnt/lustre/ |
| - | #PBS -e /mnt/lustre3p/ | + | #PBS -e /mnt/lustre/ |
| #PBS -m abe | #PBS -m abe | ||
| - | #PBS -WMail_Users=youremail@ddress | + | #PBS -M youremail@ddress |
| ulimit -s unlimited | ulimit -s unlimited | ||
| - | cd /mnt/lustre3p/ | + | cd /mnt/lustre/ |
| nproc=`cat $PBS_NODEFILE | wc -l` | nproc=`cat $PBS_NODEFILE | wc -l` | ||
| echo nproc is $nproc | echo nproc is $nproc | ||
| Line 381: | Line 384: | ||
| #PBS -q normal | #PBS -q normal | ||
| #PBS -l walltime=4: | #PBS -l walltime=4: | ||
| - | #PBS -o /mnt/lustre3p/ | + | #PBS -o /mnt/lustre/ |
| - | #PBS -e /mnt/lustre3p/ | + | #PBS -e /mnt/lustre/ |
| #PBS -m abe | #PBS -m abe | ||
| - | #PBS -WMail_Users=youremail@ddress | + | #PBS -M youremail@ddress |
| ulimit -s unlimited | ulimit -s unlimited | ||
| . / | . / | ||
| - | cd /mnt/lustre3p/ | + | cd /mnt/lustre/ |
| rm wrfout* rsl* | rm wrfout* rsl* | ||
| nproc=`cat $PBS_NODEFILE | wc -l` | nproc=`cat $PBS_NODEFILE | wc -l` | ||
| Line 407: | Line 410: | ||
| Note that in the above job script example the working directory is on the Lustre file system. | Note that in the above job script example the working directory is on the Lustre file system. | ||
| < | < | ||
| - | /mnt/lustre3p/ | + | /mnt/lustre/ |
| </ | </ | ||
| where '' | where '' | ||
/app/dokuwiki/data/attic/quick/start.1637660161.txt.gz · Last modified: 2021/12/09 16:42 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
