guide:tensorflow
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
guide:tensorflow [2020/11/03 19:37] kevin |
guide:tensorflow [2025/04/23 14:19] (current) kevin |
||
|---|---|---|---|
| Line 3: | Line 3: | ||
| If you are making use of Jupyter notebook to write your python scripts then you first need to make sure you export the .py file from Jupyter and then copy it onto the cluster | If you are making use of Jupyter notebook to write your python scripts then you first need to make sure you export the .py file from Jupyter and then copy it onto the cluster | ||
| - | Also ensure job is copied to / | + | Also ensure job is copied to '' |
| ===== Running Tensorflow on CPU nodes ===== | ===== Running Tensorflow on CPU nodes ===== | ||
| Line 16: | Line 16: | ||
| | | ||
| - | Then | + | Then '' |
| - | + | ||
| - | cd /mnt/lustre/ | + | |
| Finally run | Finally run | ||
| Line 37: | Line 35: | ||
| As with CPU version you can test your python jobs on an interactive node: | As with CPU version you can test your python jobs on an interactive node: | ||
| - | qsub -I -P YOURPROGRAMME(E.G. CSCI1234) -q gpu_1 -l select=1: | + | qsub -I -P YOURPROGRAMME(E.G. CSCI1234) -q gpu_1 -l select=1: |
| Once on an interactive node (gpuNNNN) you need to load up appropriate modules: | Once on an interactive node (gpuNNNN) you need to load up appropriate modules: | ||
| | | ||
| - | | + | |
| To test that Python (and Tensorflow) sees the GPUs: | To test that Python (and Tensorflow) sees the GPUs: | ||
| Line 48: | Line 46: | ||
| < | < | ||
| module purge | module purge | ||
| - | module load chpc/ | + | module load chpc/ |
| + | module load chpc/ | ||
| python3 | python3 | ||
| import tensorflow as tf | import tensorflow as tf | ||
| Line 85: | Line 84: | ||
| Now you can try your Python code: | Now you can try your Python code: | ||
| - | cd /mnt/lustre/ | + | cd /mnt/lustre3p/ |
| When running on a single GPU you need to include the following in your .py file to ensure that not all the CPU's on the node get consumed, thereby resulting in your job being killed by the scheduler | When running on a single GPU you need to include the following in your .py file to ensure that not all the CPU's on the node get consumed, thereby resulting in your job being killed by the scheduler | ||
| Line 99: | Line 98: | ||
| Once you are on the login node just type: | Once you are on the login node just type: | ||
| + | |||
| + | **module purge** | ||
| + | |||
| + | **module load chpc/ | ||
| **qtensorflow_cpu** (CPU version of Tensorflow) | **qtensorflow_cpu** (CPU version of Tensorflow) | ||
| Line 136: | Line 139: | ||
| Please take note of empty space in EXAMPLE2. This corresponds to the enter key | Please take note of empty space in EXAMPLE2. This corresponds to the enter key | ||
| + | ===== Updating Tensorflow on GPU nodes ===== | ||
| + | First get onto a GPU node: | ||
| + | |||
| + | qsub -I -P YOURPROGRAMME(E.G. CSCI1234) -q gpu_1 -l select=1: | ||
| + | |||
| + | To update tensorflow you should use a conda environment | ||
| + | |||
| + | module purge | ||
| + | module load chpc/ | ||
| + | |||
| + | Create your environment | ||
| + | |||
| + | conda create -n tf-gpu tensorflow-gpu python=3.8 | ||
| + | conda activate tf-gpu | ||
| + | |||
| + | The above only needs to be done once thereafter all you will need to do is: | ||
| + | module purge | ||
| + | module load chpc/ | ||
| + | conda activate tf-gpu | ||
| + | |||
| + | in order to use your updated version of tensorflow | ||
/app/dokuwiki/data/attic/guide/tensorflow.1604425067.txt.gz · Last modified: 2021/12/09 16:42 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
