OP2 on CX1

This page gives a detailed tutorial on how to build the OP2 libraries on the Imperial College CX1 compute cluster and run the provided example applications.

Building OP2 on cx1

CX1 runs Red Hat Enterprise 6, uses environment modules and PBS.

Environment

Create a file cx1.env in the root of your OP2-Common clone:

#!/bin/bash

module unload cuda
module unload intel-suite
module unload libscotch
module unload mpi
module unload parmetis

module load cuda/4.0.17
module load intel-suite/11.1
module load libscotch/5.1.12b
module load mpi
module load parmetis

export CUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME
export PARMETIS_DIR=$PARMETIS_HOME
export SCOTCH_DIR=$LIBSCOTCH_HOME

export CC=mpicc
export CXX=mpicxx

unset LD_PRELOAD

Sourcing this file (. cx1.env) sets up your bash environment as required to build OP2.

Building the OP2 C library

Create the following script as cx1.sh in the op2/c directory of your OP2-Common clone:

#!/bin/bash

. ../../cx1.env

# FindMPI.cmake cannot properly detect MPI on CX1, hence we need to set _both_
# MPI_INCLUDE_PATH and MPI_LIBRARY to disable auto-detection
./cmake.local -DBUILD_SHARED_LIBS=OFF \
  -DMPI_INCLUDE_PATH=$MPI_INCLUDE \
  -DMPI_LIBRARY=$MPI_HOME/lib/libmpi.a

Execute the script for a standard developer build of the OP2 C libraries in the current directory.

Building the OP2 C applications

Create the following script as cx1.sh in the apps/c directory of your OP2-Common clone:

#!/bin/bash

. ../../cx1.env

./cmake.local

Execute the script for a standard developer build of the OP2 C applications.