wiki.chpc.ac.za

LIGGGHTS and lammps at the CHPC

The LIGGGHTS descreet element method code is installed at the CHPC, although only limited support is available for them at this point. LIGGGHTS version 3.8 is at /apps/chpc/compmech/DEM/LIGGGHTS-3.8, but you can access it by loading the module:

module add chpc/compmech/LIGGGHTS/3.8_gcc830

At present this does not support VTK output, though this will be corrected in the near future. A module for the older version 3.6 is also available.

An example script for lights, parallelized across three nodes (select = 3) with 24 processors per node (ncpus =24:mpiprocs=24) is shown below:

liggghts.qsub

  #!/bin/bash
  #PBS -P projectid
  #PBS -l select=3:ncpus=24:mpiprocs=24:mem=12GB:nodetype=haswell_reg
  #PBS -q normal
  #PBS -l walltime=01:00:00
  #PBS -o /mnt/lustre3p/users/username/liggghtsJobs/job01/stdout
  #PBS -e /mnt/lustre3p/users/username/liggghtsJobs/job01/stderr
  #PBS -m abe
  #PBS -M username@email.co.za
  ### load the LIGGGHTS module
  module add chpc/compmech/LIGGGHTS/3.8_gcc830
  ##### Running commands
  # The -d option used with moab no longer works.  Set this environment variable explicitly.
  export PBS_JOBDIR=/mnt/lustre3p/users/username/liggghtsJobs/job01
  # Explicitly change to the job directory
  cd $PBS_JOBDIR
  nproc=`cat $PBS_NODEFILE | wc -l`
  mpirun -np $nproc liggghts < in.lj > ligghtsrun.out