ALCF Vesta

Set up your build environment

Please place the following lines in your ~/.soft file:

	@default
	+hpctoolkit
	+cmake
	+mipwrapper-gcc

Install the required Third Party Libraries (TPLs) for Trilinos

These install directions assume that a base install directory of /home/ghansen/FASTMath/src/ is used

Building ZLib

Version 1.2.8

  cd /home/ghansen/FASTMath/src/
  wget http://zlib.net/zlib-1.2.8.tar.gz
  tar -xvf zlib-1.2.8.tar.gz
  cd zlib-1.2.8
  CC=mpicc CXX=mpicxx CFLAGS=-O3 CXXFLAGS=-O3 ./configure --64 --static --prefix=/home/ghansen/FASTMath
  make install

Building Lapack

Version 3.4.2

  cd lapack-3.4.2
  mkdir build
  ./do-lapack
  make
  make install

do-lapack is the following shell script

  #!/bin/sh
  INSTALL_DIR=${HOME}/barn-shared/gcc
  cmake28 \
  -D CMAKE_BUILD_TYPE:STRING=RELEASE \
  -D CMAKE_INSTALL_PREFIX:PATH=${INSTALL_DIR} \
  -D CMAKE_Fortran_COMPILER:FILEPATH=/bgsys/drivers/ppcfloor/gnu-linux/bin/powerpc64-bgq-linux-gfortran \
  -D CMAKE_Fortran_FLAGS:STRING="-fno-underscoring" \
  ../

Building HDF5

Version 1.8.13

  cd /home/ghansen/FASTMath/src/
  wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.13.tar.gz
  tar -xvf hdf5-1.8.13.tar.gz
  cd hdf5-1.8.13
  ./configure CC=mpicc FC=mpif90 CXX=mpicxx CXXFLAGS="-fPIC -O3" CFLAGS="-fPIC -O3" FCFLAGS="-fPIC -O3" --enable-parallel --enable-shared=no --with-zlib=/home/ghansen/FASTMath --prefix=/home/ghansen/FASTMath
  make -j 8
  make install

Building Netcdf

Version of the day from the git repo

  cd /home/ghansen/FASTMath/src/
  git clone https://github.com/Unidata/netcdf-c.git
  cd netcdf-c
  autoreconf -i -f

Edit the file include/netcdf.h. Make the following edits near line 228:

	#define NC_MAX_DIMS    65536    /* max dimensions per file */
	#define NC_MAX_ATTRS    8192    
	#define NC_MAX_VARS   524288    /* max variables per file */
	#define NC_MAX_NAME   256 
	#define NC_MAX_VAR_DIMS      8     /* max per variable dimensions */

Now, configure and build:

    ./configure --prefix=/home/ghansen/FASTMath CC=mpicc FC=mpif90 CXX=mpicxx CFLAGS="-I/home/ghansen/FASTMath/include -O3" LDFLAGS="-L/home/ghansen/FASTMath/lib -O3" --disable-fsync --disable-cdmremote --disable-dap --disable-shared --disable-doxygen --enable-netcdf-4 --enable-dynamic-loading
    make -j 8 
    make install

Building boost

Version 1.53.0

Build boost with the commands

  echo "using gcc : 4.4.6 : /bgsys/drivers/ppcfloor/gnu-linux/bin/powerpc64-bgq-linux-g++ ;" >> ./tools/build/v2/user-config.jam
  echo "using mpi : /bgsys/drivers/ppcfloor/comm/gcc/bin/mpicxx ;" >> ./tools/build/v2/user-config.jam
  ./bootstrap.sh --with-libraries=signals,regex,filesystem,system,mpi,serialization,thread,program_options,exception --prefix=${HOME}/barn-shared/gcc
  ./b2 -j 4
  ./b2 -j 4 install

Building ParMetis

Version 4.0.3
Load the parmetis module
```
  module load parmetis/gnu/4.0.2
```

SuperLU

Version 4.3

  cd /home/ghansen/FASTMath/src
  wget http://crd-legacy.lbl.gov/~xiaoye/SuperLU/superlu_4.3.tar.gz
  tar -xvf superlu_4.3.tar.gz
  cd superlu_4.3

  mkdir /home/ghansen/FASTMath/SuperLU_4.3
  mkdir /home/ghansen/FASTMath/SuperLU_4.3/include
  mkdir /home/ghansen/FASTMath/SuperLU_4.3/lib

Edit make.inc for your machine and environment

  PLAT = _linux
  SuperLUroot = /home/ghansen/FASTMath/SuperLU_4.3
  SUPERLULIB      = $(SuperLUroot)/lib/libsuperlu_4.3.a
  TMGLIB          = libtmglib.a
  BLASDEF     = -DUSE_VENDOR_BLAS
  BLASLIB     = -L/soft/libraries/alcf/current/gcc/BLAS64/lib -lblas
  LIBS        = $(SUPERLULIB) $(BLASLIB)
  ARCH         = ar
  ARCHFLAGS    = cr
  RANLIB       = ranlib
  CC           = mpicc
  CFLAGS       = -DPRNTlevel=0 -O3
  NOOPTS       =
  FORTRAN      = mpif90
  FFLAGS       = -O3
  LOADER       = $(CC)
  LOADOPTS     =
  CDEFS        = -DNoChange
  MATLAB       =

Build SuperLU

  make -j 4
  cd SRC
  cp *.h /home/ghansen/FASTMath/SuperLU_4.3/include

Build Trilinos

Download Trilinos

	cd /home/ghansen/FASTMath/src
	git clone https://software.sandia.gov/trilinos/repositories/publicTrilinos

Download SCOREC

	cd /home/ghansen/FASTMath/src/publicTrilinos
	git clone https://github.com/SCOREC/core.git SCOREC

Configure the Trilinos build

	mkdir build
	cd build
	./do-configure

Example `do-configure` script executed above

	#!/bin/sh
	#
	export TRILINOS_HOME=/home/ghansen/FASTMath/src/publicTrilinos
	BUILD_DIR=`pwd`
	INSTALL_DIR=/home/ghansen/FASTMath/Trilinos/MPI_REL
	BOOST_DIR=/soft/libraries/boost/current/cnk-gcc/current
	MPI_BASE_DIR=/soft/compilers/wrappers/gcc
	NETCDF=/home/ghansen/FASTMath
	HDFDIR=/home/ghansen/FASTMath
	PARMETISDIR=/soft/libraries/alcf/current/gcc/PARMETIS
	METISDIR=/soft/libraries/alcf/current/gcc/METIS
	BLAS=/soft/libraries/alcf/current/gcc/BLAS64
	LAPACK=/soft/libraries/alcf/current/gcc/LAPACK64
	
	export BOOST_ROOT=$BOOST_DIR
	EXTRA_ARGS=$@
	
	cmake \
	-D Trilinos_EXTRA_REPOSITORIES:STRING=SCOREC \
	-D Trilinos_CONFIGURE_OPTIONS_FILE:FILEPATH=$TRILINOS_HOME/sampleScripts/AlbanySettings.cmake \
	-D CMAKE_BUILD_TYPE:STRING=NONE \
	-D TPL_FIND_SHARED_LIBS:BOOL=OFF \
	\
	-D CMAKE_C_FLAGS:STRING="-O3 -ffast-math -funroll-loops -DCODE_MP -DDISABLE_MPIIO -DNO_FILE_LOCK -DTEMPLATE_FRIENDS_NOT_SUPPORTED -DMPICH_SKIP_MPICXX -DNDEBUG -DHAVE_CONFIG_H -I${METISDIR}/include" \
	-D CMAKE_CXX_FLAGS:STRING="-O3 -ffast-math -funroll-loops -DCODE_MP -DDISABLE_MPIIO -DNO_FILE_LOCK -DTEMPLATE_FRIENDS_NOT_SUPPORTED -DMPICH_SKIP_MPICXX -DNDEBUG -DHAVE_CONFIG_H -I${METISDIR}/include" \
	-D CMAKE_Fortran_FLAGS:STRING="-O3 -ffast-math -funroll-loops" \
	\
	-D TPL_ENABLE_MPI:BOOL=ON \
	-D MPI_BASE_DIR:PATH=$MPI_BASE_DIR \
	-D Trilinos_EXTRA_LINK_FLAGS:STRING="-ldl" \
	\
	-D CMAKE_VERBOSE_MAKEFILE:BOOL=OFF \
	-D Trilinos_VERBOSE_CONFIGURE:BOOL=OFF \
	\
	-D TPL_ENABLE_Matio:BOOL=OFF \
	-D SEACAS_ENABLE_SEACASSVDI:BOOL=OFF \
	-D Trilinos_ENABLE_SEACASFastq:BOOL=OFF \
	-D Trilinos_ENABLE_SEACASBlot:BOOL=OFF \
	-D Trilinos_ENABLE_SEACASPLT:BOOL=OFF \
	-D TPL_ENABLE_X11:BOOL=OFF \
	\
	-D Boost_INCLUDE_DIRS:PATH=$BOOST_DIR/include \
	-D Boost_LIBRARY_DIRS:PATH=$BOOST_DIR/lib \
	-D BoostAlbLib_INCLUDE_DIRS:PATH=$BOOST_DIR/include \
	-D BoostAlbLib_LIBRARY_DIRS:PATH=$BOOST_DIR/lib \
	\
	-D Netcdf_INCLUDE_DIRS:PATH="${NETCDF}/include" \
	-D Netcdf_LIBRARY_DIRS:PATH="${NETCDF}/lib" \
	-D TPL_Netcdf_LIBRARIES:STRING="/home/ghansen/FASTMath/lib/libnetcdf.a;/home/ghansen/FASTMath/lib/libhdf5_hl.a;/home/ghansen/FASTMath/lib/libhdf5.a;/home/ghansen/FASTMath/lib/libz.a;/usr/lib64/libdl.a" \
	-D HDF5_INCLUDE_DIRS:PATH="${HDFDIR}/include" \
	-D HDF5_LIBRARY_DIRS:PATH="${HDFDIR}/lib" \
	\
	-D TPL_ENABLE_Zlib:STRING=ON \
	-D Zlib_INCLUDE_DIRS:PATH="${HDFDIR}/include" \
	-D Zlib_LIBRARY_DIRS:PATH="${HDFDIR}/lib" \
	\
	-D ParMETIS_INCLUDE_DIRS:PATH="${PARMETISDIR}/include" \
	-D ParMETIS_LIBRARY_DIRS:PATH="${PARMETISDIR}/lib" \
	\
    -D TPL_ENABLE_SuperLU:STRING=ON \
    -D SuperLU_INCLUDE_DIRS:PATH="/home/ghansen/FASTMath/SuperLU_4.3/include" \
    -D SuperLU_LIBRARY_DIRS:PATH="/home/ghansen/FASTMath/SuperLU_4.3/lib" \
	\
	-D TPL_ENABLE_BLAS:STRING=ON \
	-D TPL_BLAS_LIBRARIES:FILEPATH="$BLAS/lib/blas.a" \
	-D TPL_ENABLE_LAPACK:STRING=ON \
	-D LAPACK_LIBRARY_DIRS:FILEPATH="$LAPACK/lib" \
	\
	-D Zoltan_ENABLE_ULONG_IDS:BOOL=ON \
	\
	-D Trilinos_ENABLE_SCOREC:BOOL=ON \
	-D SCOREC_DISABLE_STRONG_WARNINGS:BOOL=ON \
	\
    -D Trilinos_ENABLE_ThyraTpetraAdapters:BOOL=ON \
    -D Trilinos_ENABLE_Ifpack2:BOOL=ON \
    -D Trilinos_ENABLE_Amesos2:BOOL=ON \
    -D Trilinos_ENABLE_MueLu:BOOL=ON \
    -D Teuchos_ENABLE_COMPLEX:BOOL=OFF \
	\
	-D CMAKE_INSTALL_PREFIX:PATH=${INSTALL_DIR} \
	\
	$EXTRA_ARGS \
	${TRILINOS_HOME}

Build Trilinos
```
  make -j 8
  make install
```

Build Albany

CMake configure script:

  #!/bin/sh
  #
  export ALB_ENABLE_SCOREC=ON
  export ALB_ENABLE_LCM=ON
  export ALB_ENABLE_MOR=OFF
  export ALB_ENABLE_SPECULATIVE=OFF
  export ALB_ENABLE_LAME=OFF
  export ALB_ENABLE_SG_MP=OFF
  export ALB_ENABLE_INSTALL=ON
  export TRILINOS_INSTALL_DIR=${HOME}/barn-shared/Trilinos
  export ALBANY_INSTALL_DIR=${HOME}/barn-shared/Albany

  cmake28 \
  -D ALBANY_TRILINOS_DIR:FILEPATH="$TRILINOS_INSTALL_DIR" \
  -D ENABLE_LCM:BOOL=${ALB_ENABLE_LCM} \
  -D ENABLE_LCM_SPECULATIVE:BOOL=${ALB_ENABLE_SPECULATIVE} \
  -D ENABLE_SCOREC:BOOL=${ALB_ENABLE_SCOREC} \
  -D ENABLE_MOR:BOOL=${ALB_ENABLE_MOR} \
  -D ENABLE_SG_MP:BOOL=${ALB_ENABLE_SG_MP} \
  -D ENABLE_INSTALL:BOOL=${ALB_ENABLE_INSTALL} \
  -D CMAKE_INSTALL_PREFIX:FILEPATH=${ALBANY_INSTALL_DIR} \
  -D ALBANY_MPI_OPTIONS:BOOL=ON \
  -D ALBANY_MPI_EXEC:STRING="srun" \
  -D ALBANY_MPI_EXEC_NUMPROCS_FLAG:STRING="-n" \
  -D ALBANY_MPI_EXEC_MAX_NUMPROCS:STRING="4" \
  -D ALBANY_MPI_LEADING_OPTIONS:STRING="--output=test_output.out;--time=10" \
  \
  ../

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ALCF Vesta

Set up your build environment

Install the required Third Party Libraries (TPLs) for Trilinos

Building ZLib

Building Lapack

Building HDF5

Building Netcdf

Building boost

Building ParMetis

SuperLU

Build Trilinos

Download Trilinos

Download SCOREC

Configure the Trilinos build

Example `do-configure` script executed above

Build Albany

Docker Image

SPACK Package

ToDo List

Albany FAQs

Capabilities

Sub-Projects

Compiling

Nightly Tests

Code HowTo

User Group Meetings

PAALS Tutorials

Wiki HowTo

Visualization Utilities

Good Commits

Python and Albany

Clone this wiki locally

ALCF Vesta

Set up your build environment

Install the required Third Party Libraries (TPLs) for Trilinos

Building ZLib

Building Lapack

Building HDF5

Building Netcdf

Building boost

Building ParMetis

SuperLU

Build Trilinos

Download Trilinos

Download SCOREC

Configure the Trilinos build

Example do-configure script executed above

Build Albany

Clone this wiki locally

Example `do-configure` script executed above