Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and improve anomaly forcings for ISSP cases #292

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions cime_config/stream_cdeps.py
Original file line number Diff line number Diff line change
Expand Up @@ -333,6 +333,11 @@ def create_stream_xml(
),
)
if var_key in valid_values:

# # Handle, e.g., 'bilinear' in namelist instead of bilinear (no quotes)
mod_dict[var_key] = re.sub(r"[\"\']", "", mod_dict[var_key])

# Check that key is valid
expect(
mod_dict[var_key] in valid_values[var_key],
"{} can only have values of {} for stream {} in file {}".format(
Expand Down
18 changes: 17 additions & 1 deletion datm/cime_config/buildnml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
# pylint: disable=wildcard-import,unused-wildcard-import,wrong-import-position

import os, sys
import re

_CDEPS_CONFIG = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir,os.pardir,"cime_config")
_CIMEROOT = os.environ.get("CIMEROOT")
Expand Down Expand Up @@ -115,6 +116,7 @@ def _create_namelists(case, confdir, inst_string, infile, nmlgen, data_list_path
atm_grid = case.get_value("ATM_GRID")
model_grid = case.get_value("GRID")
comp_lnd = case.get_value("COMP_LND")
compset = case.get_value("COMPSET")

# Check for incompatible options.
if "CLM" in datm_mode and comp_lnd == "clm":
Expand All @@ -135,6 +137,7 @@ def _create_namelists(case, confdir, inst_string, infile, nmlgen, data_list_path
logger.debug("DATM preso3 mode is {}".format(datm_preso3))
logger.debug("DATM topo mode is {}".format(datm_topo))
logger.debug("CLM_USRDAT_NAME is {}".format(clm_usrdat_name))
logger.debug("COMPSET is {}".format(compset))

# Initialize namelist defaults
config = {}
Expand Down Expand Up @@ -177,6 +180,17 @@ def _create_namelists(case, confdir, inst_string, infile, nmlgen, data_list_path

nmlgen.init_defaults(infile, config)

# Get anomaly forcing
# First, look in namelist
anomaly_forcing = nmlgen.get_value("anomaly_forcing")
if not anomaly_forcing or anomaly_forcing[0] is None:
# If not in namelist, check whether it's an SSP compset
ssp = re.search(r"^SSP\d+_DATM", compset)
if ssp:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand this -- it will ensure anomaly_forcing is forced to be on for SSP compsets. But, I don't think that's what we want. I think that the default certainly should be for it to be on, but the user should be able to turn it off if they want. The AF forcing is also specific to the DATM_MODE so it should only do this depending on both the compset and DATM_MODE. And since the DATM_MODE the AF forcing matches will be changing, it would be better for this to be in the XML data rather than in the python code. Better to be data driven when possible.

As such I'm thinking this part should be reverted.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See below for what I suggest to add to the namelist_definition file. We will also need to make sure compset is a valid field in the namelist defintion file.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The anomaly_forcing option none is designed to handle situations where the user wants to turn off anomaly forcing for an SSP compset. See diff below:

anomaly_forcing = nmlgen.get_value("anomaly_forcing")
-    if anomaly_forcing[0] is not None:
+    if anomaly_forcing[0] is not None and anomaly_forcing[0] != "none":
        streamlist += anomaly_forcing

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confirmed by comparing the namelists from the CTSM branch's tests without and with the new clm-noAnomalyForcing testdef:

suitedir="/glade/derecho/scratch/samrabin/tests_0219-152958de"
cd "${suitedir}"
testdir_anom_on="SMS_D_Ld5.f10_f10_mg37.ISSP245Clm50BgcCrop.derecho_intel.clm-default.0219-152958de_int"
testdir_anom_off="${testdir_anom_on/clm-default/clm-default--clm-noAnomalyForcing}"
cd "${testdir_anom_on}/CaseDocs"
for f in *; do
   echo $f
   diff $f ../../${testdir_anom_off}/CaseDocs/$f
   echo " "; echo " "
done

Relevant outputs:

datm_in
12c12
<   anomaly_forcing = "Anomaly.Forcing.cmip6.ssp245"
---
>   anomaly_forcing = 'none'


datm.streams.xml
698,726d697
<
<   <stream_info name="Anomaly.Forcing.cmip6.ssp245">
<    <taxmode>cycle</taxmode>
<    <tintalgo>nearest</tintalgo>
<    <readmode>single</readmode>
<    <mapalgo>bilinear</mapalgo>
<    <dtlimit>1.5</dtlimit>
<    <year_first>2015</year_first>
<    <year_last>2100</year_last>
<    <year_align>2015</year_align>
<    <vectors>null</vectors>
<    <meshfile>/glade/campaign/cesm/cesmdata/inputdata/share/meshes/fv0.9x1.25_141008_polemod_ESMFmesh.nc</meshfile>
<    <lev_dimname>null</lev_dimname>
<    <datafiles>
<       <file>/glade/campaign/cesm/cesmdata/inputdata/atm/datm7/anomaly_forcing/CMIP6-SSP2-4.5/af.allvars.CESM.SSP2-4.5.2015-2100_c20220628.nc</file>
<    </datafiles>
<    <datavars>
<       <var>huss  Sa_shum_af</var>
<       <var>pr    Faxa_prec_af</var>
<       <var>ps    Sa_pbot_af</var>
<       <var>rlds  Faxa_lwdn_af</var>
<       <var>rsds  Faxa_swdn_af</var>
<       <var>tas   Sa_tbot_af</var>
<       <var>uas   Sa_u_af</var>
<       <var>vas   Sa_v_af</var>
<    </datavars>
<    <offset>0</offset>
<   </stream_info>
<

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekluzek Pinging, as discussed.

ssp = ssp.group().replace("_DATM", "")
anomaly_forcing = ["Anomaly.Forcing.cmip6." + ssp.lower()]
nmlgen.set_value("anomaly_forcing", anomaly_forcing)

# Generate datm_in
namelist_file = os.path.join(confdir, "datm_in")
nmlgen.write_output_file(namelist_file, data_list_path, groups=['datm_nml','const_forcing_nml'])
Expand Down Expand Up @@ -205,8 +219,10 @@ def _create_namelists(case, confdir, inst_string, infile, nmlgen, data_list_path
bias_correct = nmlgen.get_value("bias_correct")
if bias_correct is not None:
streamlist.append(bias_correct)

# Set anomaly forcing in datm.streams.xml
anomaly_forcing = nmlgen.get_value("anomaly_forcing")
if anomaly_forcing[0] is not None:
if anomaly_forcing[0] is not None and anomaly_forcing[0] != "none":
streamlist += anomaly_forcing

# Generate datm.streams.xml
Expand Down
2 changes: 1 addition & 1 deletion datm/cime_config/namelist_definition_datm.xml
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@
<type>char(10)</type>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Near the top of the file streamslist is being set with a setting for GSWP3 like this:

      <value datm_mode="CLMGSWP3v1">
        CLMGSWP3v1.Solar,CLMGSWP3v1.Precip,CLMGSWP3v1.TPQW
      </value>

This should be expanded to have an option for GSWP3 forcing and SSP compsets. So something like:

      <value datm_mode="CLMGSWP3v1" compset_period="SSP585">
        CLMGSWP3v1.Solar,CLMGSWP3v1.Precip,CLMGSWP3v1.TPQW,Anomaly.Forcing.cmip6.ssp585
      </value>

There might be a way to append just the AF file to the end, which would be preferred. But, we'd need to figure that out. The matching needs to be figured out to make sure it works right. And of course all the SSP options need to be added.

Note that compset_period needs to be added to the buildnml file which I'll sketch out below.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes to buildnml:

buildnml needs to:

  • Get the compset from the case
  • Extract out the period part of the compset
  • Send compset_period to the config object

So something like...

    caseroot = case.get_value("CASEROOT")
    datm_mode = case.get_value("DATM_MODE")
    datm_topo = case.get_value("DATM_TOPO")
    compset = case.get_value("COMPSET")
.
.
.
    # Do some reg-ex or other magic to get the compset_period from it
    compset_period = something(compset)

    config['datm_mode'] = datm_mode
    config['datm_co2_tseries'] = datm_co2_tseries
    config['compset_period'] compset_period
.
.
.

<category>datm</category>
<group>datm_nml</group>
<valid_values>Anomaly.Forcing.Precip,Anomaly.Forcing.Temperature,Anomaly.Forcing.Pressure,Anomaly.Forcing.Humidity,Anomaly.Forcing.Uwind,Anomaly.Forcing.Vwind,Anomaly.Forcing.Shortwave,Anomaly.Forcing.Longwave</valid_values>
<valid_values>none,Anomaly.Forcing.cmip5.rcp45,Anomaly.Forcing.cmip6.ssp126,Anomaly.Forcing.cmip6.ssp245,Anomaly.Forcing.cmip6.ssp370,Anomaly.Forcing.cmip6.ssp585</valid_values>
<desc>
If set, include anomaly forcing streams in namelist.
</desc>
Expand Down
194 changes: 67 additions & 127 deletions datm/cime_config/stream_definition_datm.xml
Original file line number Diff line number Diff line change
Expand Up @@ -208,16 +208,13 @@
topo.cplhist

========================
optional streams anonomly forcing (turned on and set my datm_nml variable anomoly_forcing
optional streams anonomly forcing (turned on and set my datm_nml variable anomaly_forcing
========================
Anomaly.Forcing.Humidity
Anomaly.Forcing.Longwave
Anomaly.Forcing.Precip
Anomaly.Forcing.Pressure
Anomaly.Forcing.Shortwave
Anomaly.Forcing.Temperature
Anomaly.Forcing.Uwind
Anomaly.Forcing.Vwind
Anomaly.Forcing.cmip5.rcp45
Anomaly.Forcing.cmip6.ssp126
Anomaly.Forcing.cmip6.ssp245
Anomaly.Forcing.cmip6.ssp370
Anomaly.Forcing.cmip6.ssp585

========================
optional streams bias correction (turned on and set my datm_nml variable bias_correct)
Expand Down Expand Up @@ -1170,49 +1167,25 @@
</stream_entry>

<!-- =================== -->
<!-- anomoly forcing streams -->
<!-- anomaly forcing streams -->
<!-- =================== -->

<stream_entry name="Anomaly.Forcing.Precip">
<stream_entry name="Anomaly.Forcing.cmip5.rcp45">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
<meshfile>$DIN_LOC_ROOT/share/meshes/fv0.9x1.25_141008_polemod_ESMFmesh.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.pr.ccsm4.rcp45.2006-2300.nc</file>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.allvars.ccsm4.rcp45.2006-2300.nc</file>
</stream_datafiles>
<stream_datavars>
<var>huss Sa_shum_af</var>
<var>pr Faxa_prec_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>2006</stream_year_align>
<stream_year_first>2006</stream_year_first>
<stream_year_last>2300</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>nearest</tintalgo>
</stream_tintalgo>
<stream_taxmode>
<taxmode>cycle</taxmode>
</stream_taxmode>
<stream_dtlimit>
<dtlimit>1.5</dtlimit>
</stream_dtlimit>
<stream_readmode>single</stream_readmode>
</stream_entry>

<stream_entry name="Anomaly.Forcing.Temperature">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.tas.ccsm4.rcp45.2006-2300.nc</file>
</stream_datafiles>
<stream_datavars>
<var>ps Sa_pbot_af</var>
<var>rlds Faxa_lwdn_af</var>
<var>rsds Faxa_swdn_af</var>
<var>tas Sa_tbot_af</var>
<var>uas Sa_u_af</var>
<var>vas Sa_v_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand All @@ -1235,24 +1208,31 @@
<stream_readmode>single</stream_readmode>
</stream_entry>

<stream_entry name="Anomaly.Forcing.Pressure">
<stream_entry name="Anomaly.Forcing.cmip6.ssp126">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
<meshfile>$DIN_LOC_ROOT/share/meshes/fv0.9x1.25_141008_polemod_ESMFmesh.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.ps.ccsm4.rcp45.2006-2300.nc</file>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/CMIP6-SSP1-2.6/af.allvars.CESM.SSP1-2.6.2015-2100_c20220628.nc</file>
</stream_datafiles>
<stream_datavars>
<var>huss Sa_shum_af</var>
<var>pr Faxa_prec_af</var>
<var>ps Sa_pbot_af</var>
<var>rlds Faxa_lwdn_af</var>
<var>rsds Faxa_swdn_af</var>
<var>tas Sa_tbot_af</var>
<var>uas Sa_u_af</var>
<var>vas Sa_v_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>2006</stream_year_align>
<stream_year_first>2006</stream_year_first>
<stream_year_last>2300</stream_year_last>
<stream_year_align>2015</stream_year_align>
<stream_year_first>2015</stream_year_first>
<stream_year_last>2100</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>nearest</tintalgo>
Expand All @@ -1266,86 +1246,31 @@
<stream_readmode>single</stream_readmode>
</stream_entry>

<stream_entry name="Anomaly.Forcing.Humidity">
<stream_entry name="Anomaly.Forcing.cmip6.ssp245">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
<meshfile>$DIN_LOC_ROOT/share/meshes/fv0.9x1.25_141008_polemod_ESMFmesh.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.huss.ccsm4.rcp45.2006-2300.nc</file>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/CMIP6-SSP2-4.5/af.allvars.CESM.SSP2-4.5.2015-2100_c20220628.nc</file>
</stream_datafiles>
<stream_datavars>
<var>huss Sa_shum_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>2006</stream_year_align>
<stream_year_first>2300</stream_year_first>
<stream_year_last>2300</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>nearest</tintalgo>
</stream_tintalgo>
<stream_taxmode>
<taxmode>cycle</taxmode>
</stream_taxmode>
<stream_dtlimit>
<dtlimit>1.5</dtlimit>
</stream_dtlimit>
<stream_readmode>single</stream_readmode>
</stream_entry>

<stream_entry name="Anomaly.Forcing.Uwind">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.uas.ccsm4.rcp45.2006-2300.nc</file>
</stream_datafiles>
<stream_datavars>
<var>pr Faxa_prec_af</var>
<var>ps Sa_pbot_af</var>
<var>rlds Faxa_lwdn_af</var>
<var>rsds Faxa_swdn_af</var>
<var>tas Sa_tbot_af</var>
<var>uas Sa_u_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>2006</stream_year_align>
<stream_year_first>2006</stream_year_first>
<stream_year_last>2300</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>nearest</tintalgo>
</stream_tintalgo>
<stream_taxmode>
<taxmode>cycle</taxmode>
</stream_taxmode>
<stream_dtlimit>
<dtlimit>1.5</dtlimit>
</stream_dtlimit>
<stream_readmode>single</stream_readmode>
</stream_entry>

<stream_entry name="Anomaly.Forcing.Vwind">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.vas.ccsm4.rcp45.2006-2300.nc</file>
</stream_datafiles>
<stream_datavars>
<var>vas Sa_v_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>2006</stream_year_align>
<stream_year_first>2006</stream_year_first>
<stream_year_last>2300</stream_year_last>
<stream_year_align>2015</stream_year_align>
<stream_year_first>2015</stream_year_first>
<stream_year_last>2100</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>nearest</tintalgo>
Expand All @@ -1359,24 +1284,31 @@
<stream_readmode>single</stream_readmode>
</stream_entry>

<stream_entry name="Anomaly.Forcing.Shortwave">
<stream_entry name="Anomaly.Forcing.cmip6.ssp370">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
<meshfile>$DIN_LOC_ROOT/share/meshes/fv0.9x1.25_141008_polemod_ESMFmesh.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.rsds.ccsm4.rcp45.2006-2300.nc</file>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/CMIP6-SSP3-7.0/af.allvars.CESM.SSP3-7.0.2015-2100_c20220628.nc</file>
</stream_datafiles>
<stream_datavars>
<var>huss Sa_shum_af</var>
<var>pr Faxa_prec_af</var>
<var>ps Sa_pbot_af</var>
<var>rlds Faxa_lwdn_af</var>
<var>rsds Faxa_swdn_af</var>
<var>tas Sa_tbot_af</var>
<var>uas Sa_u_af</var>
<var>vas Sa_v_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>2006</stream_year_align>
<stream_year_first>2006</stream_year_first>
<stream_year_last>2300</stream_year_last>
<stream_year_align>2015</stream_year_align>
<stream_year_first>2015</stream_year_first>
<stream_year_last>2100</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>nearest</tintalgo>
Expand All @@ -1390,24 +1322,31 @@
<stream_readmode>single</stream_readmode>
</stream_entry>

<stream_entry name="Anomaly.Forcing.Longwave">
<stream_entry name="Anomaly.Forcing.cmip6.ssp585">
<stream_meshfile>
<meshfile>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/domain.permafrostRCN_P2.c2013.nc</meshfile>
<meshfile>$DIN_LOC_ROOT/share/meshes/fv0.9x1.25_141008_polemod_ESMFmesh.nc</meshfile>
</stream_meshfile>
<stream_datafiles>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/af.rlds.ccsm4.rcp45.2006-2300.nc</file>
<file>$DIN_LOC_ROOT/atm/datm7/anomaly_forcing/CMIP6-SSP5-8.5/af.allvars.CESM.SSP5-8.5.2015-2100_c20220628.nc</file>
</stream_datafiles>
<stream_datavars>
<var>huss Sa_shum_af</var>
<var>pr Faxa_prec_af</var>
<var>ps Sa_pbot_af</var>
<var>rlds Faxa_lwdn_af</var>
<var>rsds Faxa_swdn_af</var>
<var>tas Sa_tbot_af</var>
<var>uas Sa_u_af</var>
<var>vas Sa_v_af</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
<mapalgo>bilinear</mapalgo>
</stream_mapalgo>
<stream_vectors>null</stream_vectors>
<stream_year_align>2006</stream_year_align>
<stream_year_first>2006</stream_year_first>
<stream_year_last>2300</stream_year_last>
<stream_year_align>2015</stream_year_align>
<stream_year_first>2015</stream_year_first>
<stream_year_last>2100</stream_year_last>
<stream_offset>0</stream_offset>
<stream_tintalgo>
<tintalgo>nearest</tintalgo>
Expand All @@ -1421,6 +1360,7 @@
<stream_readmode>single</stream_readmode>
</stream_entry>


<!-- =================== -->
<!-- datm_mode CORE2_NYF -->
<!-- =================== -->
Expand Down
Loading