The visualization servers are intended for single-process pre- and post-processing only, as well as GUI monitoring of running jobs. These servers are NOT intended for parallel processing or running compute tasks. The system administrators will terminate processes that do not fit the above description without warning. However, it is now possible to get a Virtual Desktop on a compute node, where you can use the full capabilities of the compute node without restriction.

1. Basic ssh login : ssh login to Lengau
2. Setting up ssh keys : ssh keys
3. Getting a VNC session on a visualization node : VizNode VNC session
4. Getting a VNC session on a compute node : Compute Node VNC session

There is a dedicated visualization server called chpcviz1, and an identical backup server called chpclic1. Please note that these servers should be used for pre- and post-processing only. If the servers are being overloaded, we reserve the right to kill your processes without warning. These servers mount the Lustre and NFS file systems, have 64 GB of RAM and 2 Intel Xeon E5-2640 processors, each with 6 processor cores, as well as NVidia Tesla K20m GPU cards. For security reasons, these servers do not have IP addresses visible from outside the CHPC. However, they can be accessed via the login node by means of SSH tunneling. Although it is possible to perform remote visualization by means of X-forwarding (log in with “ssh -X”), this approach is generally too slow to be of use if the user is not on the internal network. It is therefore preferable to use a remote desktop. However, standard VNC is too slow for this and does not work properly with OpenGL, the library mostly used for 3D graphics. The visualization server has been set up with TurboVNC and VirtualGL, which gets around both of these problems. Use the following process for remote visualization:

• Log in to the system by normal means (command line ssh or PuTTY)
• From the cluster login node, log in to chpcviz1, by means of ssh (command: ssh chpcviz1)
• On chpcviz1, start up a TurboVNC server for your use. You can either leave this login session open or disconnect after starting the TurboVNC server, which will continue running untill it is manually shut down.
• From your own system, set up an ssh tunnel to forward a port on your system to the appropriate port on chpcviz1
• Use the TurboVNC client to connect to the VNC server on chpcviz1
• Run your graphics program with vglrun, or run your entire virtual desktop session with the vglrun wrapper, as per the instructions given below.
• When finished, close down the TurboVNC client, ssh into chpcviz1 again and kill the TurboVNC server session

There is a default VNC server installed on chpcviz. Do not use it. Use TurboVNC, which can be started with a command like this:

/apps/chpc/compmech/TurboVNC-2.2.5/bin/vncserver :3 -geometry 1920×1080 -depth 24

TurboVNC by default sets up a startup file that selects the lightweight window manager Fluxbox. If you are running TurboVNC on chpcviz1 or chpclic1 you can also use the friendlier window managers Mate or XFCE4.

Your $HOME/.vnc/xstartup.turbovnc file should look like this. Please ensure that this file is executable. You can do this with the command: chmod o+x $HOME/.vnc/xstartup.turbovnc

#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
# Uncomment out the next line if you to use the mate desktop
exec /bin/mate-session
# Uncomment the next line if you want to use the xfce4 desktop
#exec /bin/xfce4-session
# Uncomment the next two lines if you want to use the very lightweight window manager Fluxbox.  
#export PATH=/apps/chpc/compmech/fluxbox-1.3.7/bin:$PATH
#exec /apps/chpc/compmech/fluxbox-1.3.7/bin/startfluxbox

More than one VNC session can be run simultaneously. In this example, two other sessions are already running, therefore :3 is used to specify that this will be a session on virtual display 3. If this is unavailable, try :4, etc. Optionally specify an appropriate display resolution and colour depth, as in the above example. If you get a warning of unsupported resolution, try one of the standard resolutions. The port number used for a VNC session is 5900+n , where n is the number of the display. In this example, the VNC session will be served on port 5903.

The vncserver will continue running even after logging out. If you are no longer going to use it, please kill the server as follows:

/apps/chpc/compmech/TurboVNC-2.2.5/bin/vncserver -kill :3

where :3 should be changed to whichever display the server has been running on.

A known problem is that the VNC server is not compatible with Python-3.7. If you have a Python-3.7 module loaded in your environment, unload it first in order to start up the VNC server.

You cannot log into chpcviz1 directly from outside CHPC's network. However, it is easy to set up an ssh tunnel to it. There are a number of ways to set up such a tunnel:

This is an example of the command:

ssh -f user@lengau.chpc.ac.za -L 5903:chpcviz1:5903 -N

• Obviously change “user” to your own user-id.
• The -f option puts the ssh session in the background. If you wanted to log in anyway (to log in to chpcviz1 to start the VNC server, for example), omit this option.
• The -L option is essential for tunneling.
• The -N option prevents ssh from executing a remote command. This should also be omitted if you want an interactive session.
• 5903:chpcviz1:5903 means that port 5903 on the localhost will be forwarded to port 5903 on chpcviz1. The port number on the local host is arbitrary, you can use any number greater than 1024. However, there is some merit in using consistent values. chpcviz1:5903 is the destination port, with the port number given by 5900+n, as described above.

Any VNC client can be used to connect to the TurboVNC server on chpcviz1. However, to take full advantage of the higher speed and configuration options of TurboVNC, use the TurboVNC client as well. It can be downloaded from http://sourceforge.net/projects/virtualgl/files/TurboVNC/ . The Windows installer includes a customized version of PuTTY. Once TurboVNC has been installed, run the PuTTY in the TurboVNC installation directory. In the left pane, expand the SSH option, and click on Tunnels. Add the port number for your local host in the source port box, and chpcviz1:5903 (use the port number that your VNC server is using) in the destination box.

Now click on “Add”.

Log in with your usual user-id and password.

If you haven't done so already, install the TurboVNC client, either from http://sourceforge.net/projects/virtualgl/files/TurboVNC/ or from a repository for your version of Linux. Start the TurboVNC Viewer client, and specify localhost::5903 (or whichever local port number you have selected) as the VNC server. The documentation recommends using double colons and the full 590* number, but this also works:

Click on connect and log in with the VNC password you provided when starting the VNC server for the first time.

You should now get a remote desktop:

Clicking on the top left corner will open an “Options” menu:

Experiment with the various settings. You can trade off quality for speed. On a slow connection, use fast low quality settings to set up the scene, then request a “Lossless refresh” to get a high quality image.

3D programs (Paraview, for example) mostly use OpenGL. In a normal VNC session, OpenGL is most likely to throw an error, or at best run with software rendering. In order to take advantage of the graphics processing hardware on the server, it is necessary to run OpenGL programs with the VirtualGL “wrapper”. For example:

/opt/VirtualGL/bin/vglrun /apps/chpc/compmech/CFD/Paraview-4.3.1-Linux-64bit/bin/paraview &

will run a recent version of Paraview. If you have started up your X-windows session with the vglrun wrapper as per the first time instructions given above, it is not necessary to use it when starting the OpenGL application. The following $HOME/.vnc/xstartup.turbovnc file makes life easier.

#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
# Uncomment out the next line if you to use the mate desktop
exec /opt/VirtualGL/bin/vglrun /bin/mate-session
# Uncomment the next line if you want to use the xfce4 desktop
#exec  /opt/VirtualGL/bin/vglrun /bin/xfce4-session
# Uncomment the next two lines if you want to use the very lightweight window manager Fluxbox.  
#export PATH=/apps/chpc/compmech/fluxbox-1.3.7/bin:$PATH
#exec /opt/VirtualGL/bin/vglrun /apps/chpc/compmech/fluxbox-1.3.7/bin/startfluxbox

/apps/chpc/compmech/TurboVNC-2.2.3/bin/vncserver -kill :3

will shut down the VNC server for display 3, freeing up resources for other users.

It is now possible to run a VNC session directly on a compute node. The advantage of this is that far more compute power is available, as the standard compute nodes have 24 compute cores and either 128 GB or 64 GB of RAM. Even a 56-core, 1 TB fat node can be used this way, if your use case justifies it. The compute cores support AVX instructions, and can consequently render 3D graphics very effectively. The process is very similar to what is described above, with some important differences.

The command

qsub -X -I -l select=1:ncpus=24:mpiprocs=24:mem=120gb -l walltime=8:00:00 -q smp -P MECH1234

will get you a full compute node for 8 hours.

Make sure that you have a suitable $HOME/.vnc/xstartup.turbovnc file. The TurboVNC installation has been customized to generate a suitable version of this file be default when it is run for the first time, but if you have difficulties, ensure that it looks like this:

#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
export PATH=/apps/chpc/compmech/fluxbox-1.3.7/bin:$PATH
exec /apps/chpc/compmech/fluxbox-1.3.7/bin/startfluxbox

This file loads up the minimalist window manager Fluxbox, rather than a full desktop. Fluxbox also works well on the standard visualization nodes, but the full desktop is not available on compute nodes, hence the requirement for a very lightweight window manager.

Now launch the VNC server:

/apps/chpc/compmech/TurboVNC-2.2.3/bin/vncserver :1 -depth 24 -geometry 1800x1000 

Use a resolution that fits your display. Make note of the compute node hostname.

The process described above has an important vulnerability. If you lose the interactive ssh session because of a network glitch, you will also lose the VNC session. A better approach is to launch the VNC session from a normal PBS job script, which could look like this:

runVNC.qsub

#!/bin/bash
#PBS -l select=1:ncpus=4
#PBS -q serial
#PBS -l walltime=2:00:00
#PBS -P MECH1234
#PBS -o $HOME/vncsession.out
#PBS -e $HOME/vncsession.err
#PBS -m abe
#PBS -M justsomeuser@gmail.com
###  This following line will write the hostname of your compute node to the file hostname.txt
hostname > hostname.txt
/apps/chpc/compmech/TurboVNC-2.2.5/bin/vncserver :1 -depth 24 -geometry 1600x900
sleep 2h
/apps/chpc/compmech/TurboVNC-2.2.5/bin/vncserver -kill :1

Obviously customise this script to suit your own circumstances. The above script will provide 4 cores for two hours. DO NOT use more than 4 cores in the VNC session. If you intend using a full compute node, use the smp queue and request 24 cores. If you need more than 2 hours, specify the walltime as well as the sleep time accordingly. Get the compute node hostname either by looking in the file hostname.txt or by by querying PBS: qstat -awu username will give you the jobnumbers of all your jobs (please substitute “username” with your OWN username). qstat -n1 jobnumber will give you the hostname of the compute node(s) that you are using for that job.

This is identical to the process as described above for a visualization node, except that you now tunnel to the compute node that you have been allocated. If you are using Windows, you can set up the tunnel with PuTTY or MobaXterm, and from Linux, use the command line as in this example:

ssh -f jblogs@lengau.chpc.ac.za -L 5901:cnode1234:5901 -N

As usual, use your username and compute node, not jblogs and cnode1234!

Now, as before, fire up your VNC client, preferably TurboVNC, and connect to port 5901 (or 1 in VNC shorthand) on localhost. Once connected, you will be confronted with a blank grey screen. Click your right mouse button to pop up a small menu, and select xterm, which will give you a small terminal. You can resize the window by holding down the alt key and the right mouse button, and dragging the mouse. Refer to the Fluxbox homepage if you need help with other operations.

Up to this point, the process has been basically identical to using a visualization node. However, the compute nodes do not support VirtualGL, as they do not have dedicated graphics cards. All is not lost however:

1. Software GUIs with menus, etc. will generally just work.
2. Software linking dynamically to OpenGL libraries will probably work well with the chpc/compmech/mesa/20.2.2_swr or chpc/compmech/mesa/19.1.2_swr modules. The currently installed Mesa-20.2.2 supports software rendering using either LLVMpipe or Intel's swr. By default the module activates the LLVMpipe method, but this is easily changed: export GALLIUM_DRIVER=swr or export GALLIUM_DRIVER=llvmpipe.
3. Older programs, such as Paraview-4.3 /apps/chpc/compmech/CFD/ParaView-4.3.1-Linux-64bit/bin/paraview work very well with the Mesa-20.2.2 implementation. More recent ParaView versions are built with –mesa-swr and –mesa-llvm support. Although these work well with X-forwarding, they don't work in the VNC environment. We are not sure why, but we are working on it.
4. Some software vendors provide binaries that are statically linked to a Mesa library. These will generally work, but may be a bit slow.
5. STARCCM+ has its own version of Mesa-SWR. Use the command line options -mesa -rr -rrthreads N , where N is the number of cores that should be used for graphics rendering. In this implementation, you can use up to 16 threads.

Historically, Mesa software rendering has been a way of getting OpenGL-enabled software to work, albeit very slowly, on hardware without dedicated graphics-processing capabilities. However, the OpenSWR framework makes full use of the sse and avx capabilities of modern CPUs to produce good rendering performance. Recent versions of the alternative LLVMpipe implementation have similar performance, and the apps/chpc/compmech/mesa/20.2.2_swr module defaults to LLVMpipe.