Building GEOS with latest modifications to GOCART for the Carbon group

Checking out and building the model

Setting up GitHub credentials

Checking out GEOS

Much of this is already documented here, only the essential steps are replicated here for convenience.

Setting up the environment

First, you need to have the correct modules to check out the code. If you are expert user, make sure you have git and mepo in your path. If you're not, execute the following to get them (as well as anything else you might need).

module purge
module use -a /discover/swdev/gmao_SIteam/modulefiles-SLES15
module load GEOSenv

Getting the code

Then, decide where you want to set up the model code. While you can check it out in your home directory, we recommend checking it out on a scratch space because it can get pretty big, especially when you are testing multiple different model versions. On NCCS, you typically want to use /discover/nobackup/${USER}. Somewhere in that folder, check out the model with

git clone -b v11.5.2 [email protected]:GEOS-ESM/GEOSgcm.git GEOSgcm-v11.5.2

Note that 11.5.2 simply happens to be the latest released tag at the time this is being written. There is nothing sacred about that tag. If you want a later tag, you can find all release versions here.

The model consists of code in several sub-repositories, by default none of which are checked out. There is a file called components.yaml in the source tree you just checked out, which contains the tags for each repository that will be checked out. This does not yet contain RRG, so open that file and append the following:

RRG:
  local: ./src/Components/@GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/@GEOSchem_GridComp/@RRG
  remote: ../RRG.git
  branch: main
  develop: develop

Open that file, and make the following three changes (you can save the original components.yaml somewhere for peace of mind if you want):

1. Under GOCART, delete the line that starts with tag and replace it with branch: feature/sbasu1/gocart_methane
2. Under MAPL, replace the tag line with branch: feature/sbasu1/verbose_netcdf_print
3. Under GEOSchem_GridComp, replace the tag line with branch: feature/sbasu1/gocart_methane

Then issue the following at the command line:

mepo clone
mepo status

The first line checks out the code of each repository at the revision specified in components.yaml, while the second line is just to confirm that the checkout/clone operation "worked". In the future, you can use mepo status to see at a glance the status of various repositories in the source tree.

Building the code

It is best to build the code on a compute node of the same architecture as the ones you will be running the model on. For this example we will be building and running on AMD Milan nodes, so get a terminal on such a compute node with

salloc --nodes=1 --constraint=mil -t 60 -A s1460 --qos debug

This gets you a terminal on a Milan node under the debug queue, which is pretty fast but has a wall clock limit of 1 hour. You could, alternatively, issue this command first thing in the morning with -t 480 and get a node for 8 hours. You will need to wait longer to get a node, but once you do, you're set for a day's worth of building and debugging.

Once you get on a compute node, go to the folder where you checked out the source tree, and just to be safe create a clean environment as follows:

module purge
cd @env
source g5_modules.sh
cd ..

Now you're ready to build. Since you're already on a compute node, no need to submit a parallel build job. Instead, issue the following commands in order:

mkdir build
cd build
cmake .. -DBASEDIR=$BASEDIR/Linux -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../install
make -j install