Differences

This shows you the differences between two versions of the page.

--- research:ml [2019/07/06 11:36]
kgovender
+++ research:ml [2021/12/09 16:42] (current)
@@ Line 1: / Line 1: @@
-If you are making use of Jupyter notebook to write your python scripts then you first need to make sure you export the .py file in Jupyter and then copy it onto the cluster
-Also ensure job is copied to /mnt/lustre/users/YOURUSERNAME or subdirectories therein
+If you are looking for information on Tensorflow, please go to [[guide:tensorflow|TensorFlow Guide]].
-===== Running Tensorflow on non-GPU nodes =====
+======Machine Learning======
-To test a job on a normal compute node first get onto an interactive CPU compute node with the following:
+The CHPC supports Machine Learning (ML) activities at the CHPC.   More than a just a simulation of AI, machine learning is now applied to modelling areas of science where traditional mathematical models struggle with the complexity and volume of data.
-     qsub -I -P YOURPROGRAMME(E.G. CSCI1234) -q smp -l select=1:ncpus=24
-Once on an interactive node (cnodeNNNN) you need to load up the appropriate modules:
-     module purge
-     module load chpc/python/3.6.1_gcc-6.3.0
-Then
-     cd /mnt/lustre/users/YOURUSERNAME or where ever you placed your .py file
-Finally run
-     python nameofyourfile.py
-===== Running Tensorflow on GPU nodes =====
-As with CPU version you can test your python jobs on an interactive node:
-     qsub -I -P YOURPROGRAMME(E.G. CSCI1234) -q gpu_1 -l select=1:ncpus=10:ngpus=1
-Once on an interactive node (gpuNNNN) you need to load up appropriate modules:
-     module purge
-     module load chpc/cuda/10.0
-     module load chpc/python/anaconda/3
-Then
-     cd /mnt/lustre/users/YOURUSERNAME or where ever you placed your .py file
-When running on a single GPU you need to include the following in your .py file to ensure that not all the CPU's on the node get consumed, thereby resulting in your job being killed by the scheduler
-      session_conf = tf.ConfigProto(intra_op_parallelism_threads=10,inter_op_parallelism_threads=10)
-      sess = tf.Session(config=session_conf)
-Finally run
-     python nameofyourfile.py
-This page will be updated once I have written some scripts that can be used to submit jobs via the PBS scheduler  **Dr Krishna Govender**

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools