Skip to content

Moving a parameter to the params file

Jump to bottom
Bill Sacks edited this page Apr 3, 2020 · 2 revisions

General overview

This page describes the process for adding a new numeric parameter to the model, or moving a hard-coded parameter out of the code and onto the parameter file.

We are currently in a bit of a state of limbo with respect to numeric parameters in CTSM. Some parameters are hard-coded, others exist on the namelist file (lnd_in, modifiable via user_nl_clm) and others exist on the NetCDF parameter file (referenced via the paramfile variable in lnd_in). For now, we would like new numeric parameters to be added to the NetCDF parameter file. In the long-term, we plan to generate this parameter file automatically based on settings in xml files and user_nl_clm, but for now the NetCDF file needs to be modified manually.

Process for adding a new parameter or extracting a hard-coded parameter

Modify the NetCDF parameter file

There are many tools available for modifying the NetCDF parameter file (python, NCO, etc.). Here we describe a low-tech solution based on ncdump and ncgen. These utilities should be available as long as you have NetCDF installed on your system. This is probably not the best process if you have many changes or additions to make, but is a simple solution that works well enough for small changes.

First, find the parameter file currently being used by the model. The easiest way to do this is to create a new case, run case.setup and preview_namelists from the case directory, then find the paramfile in CaseDocs/lnd_in. As an example, I'll use:

 paramfile = '/Users/sacks/projects/cesm-inputdata/lnd/clm2/paramdata/clm5_params.c190829.nc'

Note that there are different versions of the parameter file for each physics version. If you are just using one physics version for your work, you only need to do this for that version of the file. However, if you are using multiple physics versions (e.g., CLM45 and CLM50), then you will need to repeat this process for each version.

Next, dump the full contents of this file to a text file. In the following, -p 9,17 is important to ensure that you write full precision of real-valued values (otherwise, the new parameter file will have roundoff level differences from the original):

ncdump -p 9,17 /Users/sacks/projects/cesm-inputdata/lnd/clm2/paramdata/clm5_params.c190829.nc > params_old.cdl

Next copy this to a new file:

cp params_old.cdl params_modified.cdl

then edit params_modified.cdl.

Note that there are two sections in this file. The first gives the metadata and the second gives the data. To introduce a new variable, you will need to make additions in both sections. You can follow the example of existing variables. You should provide at least a long_name and units, and consider providing a comment (which, for example, can give the source for the value of this parameter). We keep this file in alphabetical order, so please do the same with your changes. Also, note the semicolons at the end of each line; these are important for the syntax of this file.

For example, I added the following lines in the header section:

        double my_new_variable ;
                my_new_variable:long_name = "Some informative description" ;
                my_new_variable:units = "Units for this variable" ;
                my_new_variable:comment = "Optional comment, such as the source for the value" ;

and the following in the data section:

 my_new_variable = 10.5 ;

When you are done with your edits, generate a new parameter file. It is helpful to include the name of the old parameter file so you can remember the starting point for your changes:

ncgen -o clm5_params.c190829_wjs.c200403.nc params_modified.cdl

(Here, I have added my initials, wjs, and the creation date of this new file to the end of the file name.)

Diff the old and new parameter files and save the diff

You should then examine the differences between the old and new parameter files. You can again use ncdump for this purpose, together with the diff utility:

ncdump -p 9,17 clm5_params.c190829_wjs.c200403.nc > params_new.cdl
diff -u params_old.cdl params_new.cdl > paramdiffs_wjs.c200403.patch

Examine mydiffs.patch to make sure it looks right. Save the resulting difference file (for example, alongside the parameter file) for later reference. For example, I see:

--- params_old.cdl	2020-04-03 14:17:10.000000000 -0600
+++ params_new.cdl	2020-04-03 14:49:34.000000000 -0600
@@ -1,4 +1,4 @@
-netcdf clm5_params.c190829 {
+netcdf clm5_params.c190829_wjs.c200403 {
 dimensions:
 	pft = 79 ;
 	segment = 4 ;
@@ -671,6 +671,10 @@
 		mxtmp:long_name = "Max Temperature, parameter used in accFlds" ;
 		mxtmp:units = "C" ;
 		mxtmp:_FillValue = 0. ;
+	double my_new_variable ;
+		my_new_variable:long_name = "Some informative description" ;
+		my_new_variable:units = "Units for this variable" ;
+		my_new_variable:comment = "Optional comment, such as the source for the value" ;
 	double n_melt_coef ;
 		n_melt_coef:long_name = "N_melt parameter" ;
 		n_melt_coef:units = "unitless" ;
@@ -2626,6 +2630,8 @@
     30, 30, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, 30, 30, _,
     _, _, _, 30, 30, _, _, _, _, _, _, 30, 30, 30, 30 ;

+ my_new_variable = 10.5 ;
+
  n_melt_coef = 200 ;

  ndays_off = 15 ;

Point to the new parameter file in namelist_defaults_ctsm.xml

In principle, you can just point to your new parameter file in the user_nl_clm file for each case you set up. However, we highly recommend modifying namelist_defaults_ctsm.xml to point to your new file, for two reasons:

  1. This way you won't need to remember to change user_nl_clm for every case

  2. Perhaps more importantly, you will then be notified of a conflict if you update your branch to a newer version of master and the parameter file has also changed on master. (What to do in this situation is described below.)

To do this, open bld/namelist_files/namelist_defaults_ctsm.xml, and find a line like the following:

<paramfile phys="clm5_0">lnd/clm2/paramdata/clm5_params.c190829.nc</paramfile>

Change this to point to the full path to your parameter file (starting with a '/'). (Note that, if you use a relative path like above, the assumption is that the file is in the standard CESM inputdata space.) Then commit this change to your branch.

Please do NOT commit the updated NetCDF parameter file itself to your branch.

What to do if a new parameter file has been introduced in the model

If you later update your branch to the latest version of master and there is a conflict in this line, that is a signal that you will need to reintroduce your changes on top of the latest version of the parameter file.

Start by examining the namelist_defaults_ctsm.xml file to see what file is pointed to on master (or whatever version of the code you are merging into your branch). You may not yet have this file on disk on your machine; if not, you should be able to find it in our inputdata repository: https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/lnd/clm2/paramdata/, so you can download it from there.

You can then repeat the above process. However, particularly if you have numerous changes, it can be easier to try to apply your changes automatically. To do this, you'll use the patch file that you saved earlier. For example, if the new version of the parameter file is clm5_params.c200402.nc, then you can do something like this:

ncdump -p 9,17 clm5_params.c200402.nc > params_old.cdl
patch -p0 -o params_modified.cdl < paramdiffs_wjs.c200403.patch

This will create a file named params_modified.cdl starting with params_old.cdl and applying your differences from before. Just as with merging branches in git, though, sometimes there are conflicts that cannot be resolved automatically. The patch utility will point out that there are conflicts, and will save the "rejected hunks" to a file; you can then insert just these changes manually. Note that it is normal for there to be a conflict in the very first line of the file, which gives the file name; you don't need to resolve that one.

Then regenerate the parameter file with:

ncgen -o clm5_params.c200402_wjs.c200403.nc params_modified.cdl

This new version of the file should be identical to the one on master except for your changes. Confirm this by following the process above for diffing the files and saving the diffs.

Finally, resolve the conflict in namelist_defaults_ctsm.xml by pointing to this new version of the file.

Reading your new parameter in the code

You will need to add a small amount of code to read your new parameter in the model. For parameters that are only used by a single module, our preference is to store the parameter value locally in that module, preferably inside a data structure. You can then read it in with a line like this:

    call readNcdioScalar(ncid, 'a_coef', subname, params_inst%a_coef)

(That call is appropriate for scalar variables; paramUtilMod also has routines for reading arrays.)

This is probably easiest to see by example. The file src/biogeophys/BareGroundFluxesMod.F90 provides a relatively simple example of doing this. Note that the readParams subroutine in that module is called from src/main/readParamsMod.F90. For modules that already have an init routine, their readParams routine can instead be called from their init routine.

What to do in your Pull Request

Please do NOT commit your updated parameter file to your branch. Instead, when you are ready to open a Pull Request for your changes, please make a comment about the parameter changes that are needed. An ideal way to do this is to attach the patch file that you saved earlier in a PR comment. Please also tell us the version of the parameter file that was the baseline for yours (i.e., the version of the file on master that you started from). If the new parameter file is on an NCAR machine, please also point us to the location of this file.

Clone this wiki locally