Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate use of initial increment/perturbation file from REPLAY/+03 ICs #3119

Open
wants to merge 107 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
30cf5df
initial creation of CI using replay ICs
NeilBarton-NOAA Jul 2, 2024
4b3ae39
change name
NeilBarton-NOAA Jul 2, 2024
fe2054e
making an S2SWA case
NeilBarton-NOAA Jul 2, 2024
7b7ee9b
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
NeilBarton-NOAA Jul 22, 2024
225d280
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
NeilBarton-NOAA Jul 23, 2024
b17765e
start of CI for GEFS replays # Please enter the commit message for your
NeilBarton-NOAA Jul 24, 2024
675cb77
Merge branch 'develop' into gefs_replay_ci
NeilBarton-NOAA Jul 24, 2024
e49dbcf
removing shell check errors
NeilBarton-NOAA Jul 24, 2024
ec4becc
Merge branch 'gefs_replay_ci' of https://github.com/NeilBarton-NOAA/g…
NeilBarton-NOAA Jul 24, 2024
b69d653
fixes for wave post processing
NeilBarton-NOAA Jul 24, 2024
084d744
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
NeilBarton-NOAA Aug 13, 2024
a4bb8f3
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
NeilBarton-NOAA Aug 13, 2024
4ff31fd
add BREAKPOINTS variable
NeilBarton-NOAA Aug 13, 2024
97b92ee
fix shell checks
NeilBarton-NOAA Aug 14, 2024
713f41d
edit defult config.efc to be close to re-forecasting
NeilBarton-NOAA Aug 14, 2024
af31d2f
Fix bug in variable check
EricSinsky-NOAA Aug 15, 2024
cec7b81
Add more settings to GEFS replay CI test
EricSinsky-NOAA Aug 15, 2024
4aaca9a
Add atmos variables requested for reforeacst
EricSinsky-NOAA Aug 15, 2024
11e1957
Reorganize TSNOWP in paramlist
EricSinsky-NOAA Aug 15, 2024
a005e5f
Extract ice variables from history file instead
EricSinsky-NOAA Aug 15, 2024
2dd9a7e
Fix typo
EricSinsky-NOAA Aug 15, 2024
b132408
Update TCDC variable to correct name
EricSinsky-NOAA Aug 15, 2024
3d9455b
Merge pull request #7 from EricSinsky-NOAA/gefs_replay_ci_pr
NeilBarton-NOAA Aug 20, 2024
447224c
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
NeilBarton-NOAA Aug 20, 2024
c6b7eaa
edits to stage_ic for replay perturbations
NeilBarton-NOAA Aug 21, 2024
51aa2dd
adding icsdir to pr CI yaml
NeilBarton-NOAA Aug 21, 2024
a641591
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
NeilBarton-NOAA Aug 26, 2024
4985361
Fix ensstat failures
EricSinsky-NOAA Aug 27, 2024
8eaef27
updates for products
NeilBarton-NOAA Aug 28, 2024
2e7241f
Add shellcheck disable and skip f000 for ocn/ice
EricSinsky-NOAA Aug 28, 2024
ddc544d
Revert "Fix ensstat failures"
EricSinsky-NOAA Aug 28, 2024
40515ca
Merge remote-tracking branch 'nb/gefs_replay_ci' into gefs_replay_ci_pr
EricSinsky-NOAA Aug 28, 2024
3d16fb7
Merge pull request #8 from EricSinsky-NOAA/gefs_replay_ci_pr
NeilBarton-NOAA Aug 28, 2024
5f37ce8
Merge remote-tracking branch 'nb/gefs_replay_ci' into gefs_replay_ci_pr
EricSinsky-NOAA Aug 28, 2024
9e84240
Move shellcheck disable
EricSinsky-NOAA Aug 28, 2024
fa25984
Remove shellcheck disable
EricSinsky-NOAA Aug 28, 2024
23564f1
Add double quotes in ocnice_extractvars
EricSinsky-NOAA Sep 3, 2024
2176326
Merge branch 'develop' into gefs_replay_ci
NeilBarton-NOAA Sep 10, 2024
e99b3f4
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
NeilBarton-NOAA Sep 12, 2024
154e72f
updates for new MOM6 100 input file
NeilBarton-NOAA Sep 12, 2024
57ce47b
Update workflow/rocoto/gefs_tasks.py
NeilBarton-NOAA Sep 12, 2024
b62e4ca
fix archiving until archiving PR is ready
NeilBarton-NOAA Sep 16, 2024
d393a13
Merge branch 'gefs_replay_ci' of https://github.com/NeilBarton-NOAA/g…
NeilBarton-NOAA Sep 16, 2024
8c875a9
Merge branch 'develop' into gefs_replay_ci
NeilBarton-NOAA Sep 16, 2024
a87fa6c
change back to ufs.top edits
NeilBarton-NOAA Sep 16, 2024
058ba52
Merge branch 'gefs_replay_ci' of https://github.com/NeilBarton-NOAA/g…
NeilBarton-NOAA Sep 16, 2024
3e74753
Update scripts/exgfs_wave_post_pnt.sh
NeilBarton-NOAA Sep 16, 2024
830d904
Update parm/archive/gefs_arcdir.yaml.j2
NeilBarton-NOAA Sep 17, 2024
c938275
add comments for coarse grained restarts
NeilBarton-NOAA Sep 17, 2024
f51fed8
Merge branch 'gefs_replay_ci' of https://github.com/NeilBarton-NOAA/g…
NeilBarton-NOAA Sep 17, 2024
c842090
Make archive job work with replay run
EricSinsky-NOAA Sep 19, 2024
0802ef1
Add Foundation SST to group-a products
EricSinsky-NOAA Sep 19, 2024
12e940c
Modify gefs_arcdir yaml
EricSinsky-NOAA Sep 19, 2024
66c7117
Fix gefs_arcdir yaml
EricSinsky-NOAA Sep 19, 2024
69fccdd
updates for stage_ic
NeilBarton-NOAA Sep 19, 2024
d93d094
Merge branch 'develop' into gefs_replay_ci
NeilBarton-NOAA Sep 19, 2024
788a4c4
updates for SFS 1 degree stage ICs
NeilBarton-NOAA Sep 20, 2024
446e02b
updates to run C96mx100 with interpolated MOM6 ICs
NeilBarton-NOAA Sep 24, 2024
ed8aa9d
run SFS at 1 degree
NeilBarton-NOAA Sep 25, 2024
c75540c
remove replay ci
NeilBarton-NOAA Sep 25, 2024
259665e
updates for output
NeilBarton-NOAA Sep 26, 2024
fef15c5
Merge remote-tracking branch 'origin/develop' into SFS
NeilBarton-NOAA Sep 26, 2024
5d05216
added atmos products to a
NeilBarton-NOAA Sep 26, 2024
c9ee432
point to CI ICs
NeilBarton-NOAA Sep 27, 2024
30beca0
adding hydrostatic options
NeilBarton-NOAA Oct 4, 2024
077ad5f
SFS output and auto re-submission and yamls in new directory
NeilBarton-NOAA Oct 8, 2024
7e09580
updates for products
NeilBarton-NOAA Oct 11, 2024
b019dc7
edit time step for stability
NeilBarton-NOAA Oct 11, 2024
67272c3
Merge branch 'develop' into SFS
NeilBarton-NOAA Oct 15, 2024
ff3bf91
Merge branch 'develop' into SFS
NeilBarton-NOAA Oct 15, 2024
ceeea0b
Merge branch 'SFS' of https://github.com/NeilBarton-NOAA/global-workf…
NeilBarton-NOAA Oct 16, 2024
d08d66a
Merge branch 'SFS' of https://github.com/NeilBarton-NOAA/global-workf…
NeilBarton-NOAA Oct 16, 2024
d64111d
add changes for UPP PR
NeilBarton-NOAA Oct 16, 2024
2b97e38
merge local changes
NeilBarton-NOAA Oct 16, 2024
035cf24
updates for ci tests
NeilBarton-NOAA Oct 17, 2024
8acc70a
add defaults for gfs runs
NeilBarton-NOAA Oct 17, 2024
5be5a75
pynorm fix
NeilBarton-NOAA Oct 17, 2024
e2587ae
Update ush/forecast_predet.sh
NeilBarton-NOAA Oct 17, 2024
cfa1543
Update ush/forecast_predet.sh
NeilBarton-NOAA Oct 17, 2024
35ee836
Update ush/forecast_predet.sh
NeilBarton-NOAA Oct 17, 2024
50c5b5e
edits for gefs defaults
NeilBarton-NOAA Oct 17, 2024
3c4bbbf
move DO_SKEB to config.efcs
NeilBarton-NOAA Oct 23, 2024
3e32354
Update parm/stage/master_gefs.yaml.j2
NeilBarton-NOAA Oct 23, 2024
5bf7036
Update parm/stage/master_gefs.yaml.j2
NeilBarton-NOAA Oct 23, 2024
7a6188b
fixed typo
NeilBarton-NOAA Oct 24, 2024
b5f217a
Merge branch 'SFS' of https://github.com/NeilBarton-NOAA/global-workf…
NeilBarton-NOAA Oct 24, 2024
367f1e4
update typo
NeilBarton-NOAA Oct 28, 2024
44c6d11
update gdas.cd with develop version
NeilBarton-NOAA Oct 29, 2024
6912e43
Merge branch 'develop' into SFS
NeilBarton-NOAA Oct 29, 2024
686ba4f
update modules with dev version
NeilBarton-NOAA Oct 31, 2024
df77207
add option to C48
NeilBarton-NOAA Nov 1, 2024
b3e09dc
Merge branch 'develop' into SFS
aerorahul Nov 5, 2024
4dcb3ac
remove gfs_cyc option
NeilBarton-NOAA Nov 6, 2024
2215d40
add fix for CI test
NeilBarton-NOAA Nov 8, 2024
fcc0983
Merge branch 'develop' into SFS
aerorahul Nov 13, 2024
964b6c1
updates to run with perturbation files and seperature pertubation files
NeilBarton-NOAA Nov 19, 2024
7d3e12f
RERUN should not use initial increments
NeilBarton-NOAA Nov 21, 2024
94e5870
Merge branch 'develop' into SFS
NeilBarton-NOAA Nov 21, 2024
65bbda4
fix shell check
NeilBarton-NOAA Nov 21, 2024
e99a613
fix hydrostatic option
NeilBarton-NOAA Nov 22, 2024
8b816e3
update hydrostatic options
NeilBarton-NOAA Nov 22, 2024
8bc048a
update for hydrostatic runs
NeilBarton-NOAA Nov 22, 2024
f8d5395
Update parm/config/gefs/config.base
NeilBarton-NOAA Dec 4, 2024
62948d7
Merge branch 'develop' into SFS
NeilBarton-NOAA Dec 4, 2024
a0f21cc
change KEEP data to off
NeilBarton-NOAA Dec 6, 2024
14f3ec9
updates to run 4 month forecast out-of-the-box
NeilBarton-NOAA Dec 6, 2024
6223029
Merge branch 'develop' into SFS
NeilBarton-NOAA Dec 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion ci/cases/yamls/gefs_defaults_ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ base:
ACCOUNT: {{ 'HPC_ACCOUNT' | getenv }}
SFS_POST: "NO"
FHOUT_GFS: 6
stage_ic:
USE_OCN_ENS_PERTURB_FILES: "NO"
USE_ATM_ENS_PERTURB_FILES: "NO"
ocn:
Expand Down
7 changes: 3 additions & 4 deletions ci/cases/yamls/sfs_defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ base:
DO_BUFRSND: "NO"
DO_GEMPAK: "NO"
DO_AWIPS: "NO"
KEEPDATA: "YES"
KEEPDATA: "NO"
DO_EXTRACTVARS: "NO"
FHMAX_GFS: 2976
FHMAX_HF_GFS: 0
Expand All @@ -17,6 +17,8 @@ base:
FHOUT_ICE_GFS: 24
FCST_BREAKPOINTS: ""
REPLAY_ICS: "NO"
USE_OCN_ENS_PERTURB_FILES: "YES"
USE_ATM_ENS_PERTURB_FILES: "YES"
HPSSARCH: "NO"
LOCALARCH: "NO"
SFS_POST: "YES"
Expand All @@ -26,8 +28,5 @@ fcst:
MONO: "mono"
reforecast: "YES"
FHZER: 24
stage_ic:
USE_OCN_ENS_PERTURB_FILES: "YES"
USE_ATM_ENS_PERTURB_FILES: "YES"
ocn:
MOM6_INTERP_ICS: "YES"
5 changes: 5 additions & 0 deletions parm/config/gefs/config.base
Original file line number Diff line number Diff line change
Expand Up @@ -227,8 +227,13 @@ export SDATE_GFS=@SDATE_GFS@
export REPLAY_ICS=@REPLAY_ICS@
if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
export OFFSET_START_HOUR=$(( assim_freq / 2 ))
echo "WARNING: Replay ICs require perturbation files, ignoring any previous settings"
export USE_OCN_ENS_PERTURB_FILES="YES"
NeilBarton-NOAA marked this conversation as resolved.
Show resolved Hide resolved
export USE_ATM_ENS_PERTURB_FILES="YES"
else
export OFFSET_START_HOUR=0
export USE_OCN_ENS_PERTURB_FILES=@USE_OCN_ENS_PERTURB_FILES@
export USE_ATM_ENS_PERTURB_FILES=@USE_ATM_ENS_PERTURB_FILES@
fi

# GFS output and frequency
Expand Down
2 changes: 1 addition & 1 deletion parm/config/gefs/config.efcs
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ if (( OCNRES <= 100 )); then
export EPBL_LSCALE="500.E3,1000.E3,2000.E3,2000.E3,2000.E3"
fi

if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
if [[ "${USE_OCN_ENS_PERTURB_FILES:-NO}" == "YES" ]]; then
export ODA_INCUPD="True"
export ODA_TEMPINC_VAR='t_pert'
export ODA_SALTINC_VAR='s_pert'
Expand Down
1 change: 0 additions & 1 deletion parm/config/gefs/config.fcst
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,6 @@ case ${imp_physics} in
else
export dddmp=0.
fi
export dddmp=0.1
export d4_bg=0.12
;;
11) # GFDL
Expand Down
4 changes: 0 additions & 4 deletions parm/config/gefs/config.nsst
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,6 @@ export NST_MODEL=2

# nstf_name(2) : NST_SPINUP : 0 = OFF, 1 = ON,
export NST_SPINUP=0
cdate="${PDY}${cyc}"
if (( cdate < 2017072000 )); then
export NST_SPINUP=1
fi
Comment on lines -15 to -18
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why this is being removed? No longer need?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ICs used for the reforecasting effort have NSST spun up.


WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
# nstf_name(3) : NST_RESV (Reserved, NSST Analysis) : 0 = OFF, 1 = ON
export NST_RESV=0
Expand Down
2 changes: 1 addition & 1 deletion parm/config/gefs/config.resources
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ case ${step} in

case "${CASE}" in
"C48" | "C96" | "C192")
declare -x "walltime"="03:00:00"
declare -x "walltime"="04:00:00"
;;
"C384" | "C768" | "C1152")
declare -x "walltime"="06:00:00"
Expand Down
4 changes: 2 additions & 2 deletions parm/config/gefs/config.ufs
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ case "${fv3_res}" in
export DELTIM=600
export layout_x=2
export layout_y=2
export layout_x_gfs=4
export layout_y_gfs=4
export layout_x_gfs=6
export layout_y_gfs=8
export nthreads_fv3=1
export nthreads_fv3_gfs=1
export nthreads_ufs=1
Expand Down
6 changes: 2 additions & 4 deletions parm/config/gefs/yaml/defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,22 +14,20 @@ base:
FHOUT_HF_GFS: 1
FCST_BREAKPOINTS: "48"
REPLAY_ICS: "NO"
USE_OCN_PERTURB_FILES: "false"
FHOUT_GFS: 6
FHOUT_OCN_GFS: 6
FHOUT_ICE_GFS: 6
HPSSARCH: "NO"
LOCALARCH: "NO"
SFS_POST: "NO"
USE_OCN_ENS_PERTURB_FILES: "NO"
USE_ATM_ENS_PERTURB_FILES: "NO"
DO_TEST_MODE: "NO"
fcst:
reforecast: "NO"
FHZER: 6
TYPE: "nh"
MONO: "non-mono"
stage_ic:
USE_OCN_ENS_PERTURB_FILES: "NO"
USE_ATM_ENS_PERTURB_FILES: "NO"
ocn:
MOM6_INTERP_ICS: "NO"
# config.aero has just a system-specific path to add.
Expand Down
1 change: 1 addition & 0 deletions ush/forecast_det.sh
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ UFS_det(){
MOM6_INIT_FROM_Z=True
MOM6_WARMSTART_FILE="none"
MOM6_INIT_UV="zero"
ODA_INCUPD="False"
fi

# Check for CICE6 restart availability
Expand Down
14 changes: 9 additions & 5 deletions ush/forecast_postdet.sh
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,10 @@ FV3_postdet() {
# Determine increment files when doing cold start
if [[ "${warm_start}" == ".false." ]]; then

if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
IAU_FHROT=${half_window} # Replay ICs start at the end of the assimilation window
if [[ "${USE_ATM_ENS_PERTURB_FILES:-NO}" == "YES" ]]; then
if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
IAU_FHROT=${half_window} # Replay ICs start at the end of the assimilation window
fi
if (( MEMBER == 0 )); then
inc_files=()
else
Expand All @@ -109,7 +111,7 @@ FV3_postdet() {
fi
local increment_file
for inc_file in "${inc_files[@]}"; do
increment_file="${COMIN_ATMOS_INPUT}/${RUN}.t${cyc}z.${inc_file}"
increment_file="${COMIN_ATMOS_ANALYSIS}/${RUN}.t${cyc}z.${inc_file}"
if [[ -f "${increment_file}" ]]; then
${NCP} "${increment_file}" "${DATA}/INPUT/${inc_file}"
else
Expand Down Expand Up @@ -173,8 +175,10 @@ EOF
inc_files=("atminc.nc")
read_increment=".true."
res_latlon_dynamics="atminc.nc"
if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
IAU_FHROT=${half_window} # Replay ICs start at the end of the assimilation window
if [[ "${USE_ATM_ENS_PERTURB_FILES:-NO}" == "YES" ]]; then
if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
IAU_FHROT=${half_window} # Replay ICs start at the end of the assimilation window
fi
# Control member has no perturbation
if (( MEMBER == 0 )); then
inc_files=()
Expand Down
16 changes: 8 additions & 8 deletions ush/forecast_predet.sh
Original file line number Diff line number Diff line change
Expand Up @@ -351,27 +351,27 @@ FV3_predet(){
if [[ "${TYPE}" == "nh" ]]; then # monotonic and non-hydrostatic
hord_mt=${hord_mt_nh_mono:-"10"}
hord_xx=${hord_xx_nh_mono:-"10"}
hord_dp=-${hord_xx_nh_nonmono:-"-10"}
hord_dp=${hord_xx_nh_mono:-"10"}
else # monotonic and hydrostatic
hord_mt=${hord_mt_hydro_mono:-"10"}
hord_xx=${hord_xx_hydro_mono:-"10"}
hord_dp=-${hord_xx_nh_nonmono:-"-10"}
hord_dp=${hord_xx_hydro_mono:-"10"}
kord_tm=${kord_tm_hydro_mono:-"-12"}
kord_mt=${kord_mt_hydro_mono:-"12"}
kord_wz=${kord_wz_hydro_mono:-"12"}
kord_tr=${kord_tr_hydro_mono:-"12"}
fi
else # non-monotonic options
d_con=${d_con_nonmono:-"1."}
do_vort_damp=".true."
if [[ "${TYPE}" == "nh" ]]; then # non-monotonic and non-hydrostatic
hord_mt=${hord_mt_nh_nonmono:-"5"}
hord_xx=${hord_xx_nh_nonmono:-"5"}
hord_dp=${hord_xx_hydro_mono:-"-5"}
hord_dp=${hord_xx_nh_nonmono:-"-5"}
Copy link
Contributor

@XiaqiongZhou-NOAA XiaqiongZhou-NOAA Dec 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be changed to hord_dp=${hord_dp_nh_nonmono:-"-5"}.
Then add "export hord_dp_nh_nonmono=-5" in parm/config/gfs/config.fcst
I guess have to do the same for gefs/config.fcst.

else # non-monotonic and hydrostatic
hord_mt=${hord_mt_hydro_nonmono:-"10"}
hord_xx=${hord_xx_hydro_nonmono:-"10"}
hord_dp=${hord_xx_hydro_mono:-"10"}
kord_tm=${kord_tm_hydro_mono:-"-12"}
kord_mt=${kord_mt_hydro_mono:-"12"}
kord_wz=${kord_wz_hydro_mono:-"12"}
kord_tr=${kord_tr_hydro_mono:-"12"}
hord_dp=${hord_xx_hydro_nonmono:-"10"}
fi
fi

Expand Down