Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow building of the ufs-weather-model, WW3 pre/post execs for GFS, GEFS, SFS in the same clone of global-workflow #3098

Open
wants to merge 27 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1c5b14b
allow building multiple ufs_model.x for gfs, gefs and sfs
aerorahul Nov 14, 2024
98f26a8
use SFS_POST as a proxy for SFS
aerorahul Nov 14, 2024
041e7e1
link/copy the appropriate model executable
aerorahul Nov 14, 2024
e886c38
Merge branch 'develop' into feature/multi_model_build
aerorahul Nov 25, 2024
e536a3a
create gfs/gefs versions of ww3 pre/post execs
aerorahul Dec 4, 2024
2a9f12b
Merge branch 'develop' into feature/multi_model_build
aerorahul Dec 4, 2024
b8eb15d
use NET specific executable names
aerorahul Dec 5, 2024
f0bf57c
fix errors detected by shellnorms
aerorahul Dec 5, 2024
3bd366a
one shellnorm error escaped
aerorahul Dec 5, 2024
b102c07
one shellnorm error escaped
aerorahul Dec 5, 2024
c26da40
Merge branch 'develop' into feature/multi_model_build
aerorahul Dec 5, 2024
8bee97c
minor fixes for typos and such
aerorahul Dec 5, 2024
612ef35
update Jenkinsfile to build in a single location and use it
aerorahul Dec 6, 2024
080186a
remove build matrix from Jenkinsfile
aerorahul Dec 6, 2024
f8e3d6a
fix shellnorm found issue
aerorahul Dec 6, 2024
fc41b6d
fix shellnorm found issue by disabling it
aerorahul Dec 6, 2024
c61ff88
fix not changed shellcheck detected error
aerorahul Dec 6, 2024
200d6bd
add comment back in ww3prepost build script
aerorahul Dec 6, 2024
37d1fb9
Update ush/wave_grid_interp_sbs.sh
aerorahul Dec 9, 2024
ab66442
Update sorc/link_workflow.sh
aerorahul Dec 10, 2024
dae774a
Update ush/wave_grid_interp_sbs.sh
aerorahul Dec 10, 2024
1d47c9e
Update ush/wave_prnc_cur.sh
aerorahul Dec 10, 2024
b136b8a
Update ush/wave_prnc_ice.sh
aerorahul Dec 10, 2024
ebf6176
Update scripts/exgfs_wave_post_pnt.sh
aerorahul Dec 10, 2024
93ad4bf
Update ush/wave_outp_spec.sh
aerorahul Dec 10, 2024
ea41c19
Merge branch 'develop' into feature/multi_model_build
aerorahul Dec 10, 2024
b5ede8b
capture stdout/stderr from Jenkinsbuild for gefs variants. Also, was…
aerorahul Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 77 additions & 102 deletions ci/Jenkinsfile

Large diffs are not rendered by default.

3 changes: 0 additions & 3 deletions ci/cases/yamls/build.yaml

This file was deleted.

4 changes: 2 additions & 2 deletions ci/scripts/utils/ci_utils_wrapper.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
HOMEgfs="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." >/dev/null 2>&1 && pwd )"
source "${HOMEgfs}/ush/detect_machine.sh"

utitilty_function="${1}"
utility_function="${1}"

source "${HOMEgfs}/ci/scripts/utils/ci_utils.sh"
${utitilty_function} "${@:2}"
${utility_function} "${@:2}"
10 changes: 7 additions & 3 deletions parm/config/gefs/config.fcst
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,11 @@ export esmf_logkind="ESMF_LOGKIND_MULTI_ON_ERROR" #Options: ESMF_LOGKIND_MULTI_O

export FORECASTSH="${SCRgfs}/exglobal_forecast.sh"
#export FORECASTSH="${SCRgfs}/exglobal_forecast.py" # Temp. while this is worked on
export FCSTEXEC="ufs_model.x"
if [[ "${SFS_POST:-}" == "YES" ]]; then
export FCSTEXEC="sfs_model.x"
else
export FCSTEXEC="gefs_model.x"
fi

#######################################################################
# Model configuration
Expand Down Expand Up @@ -105,7 +109,7 @@ if (( gwd_opt == 2 )); then
export do_gsl_drag_ls_bl=".true."
export do_gsl_drag_ss=".false."
export do_gsl_drag_tofd=".true."
export do_gwd_opt_psl=".true."
export do_gwd_opt_psl=".true."
export do_ugwp_v1_orog_only=".false."
launch_level=$(echo "${LEVS}/2.35" |bc)
export launch_level
Expand Down Expand Up @@ -133,7 +137,7 @@ if [[ "${progsigma}" == ".true." ]]; then tbp="_progsigma" ; fi
if [[ "${DO_AERO_FCST}" == "YES" ]]; then
export IAER=2011 # spectral band mapping method for aerosol optical properties
else
export IAER=1011
export IAER=1011
fi
export iovr_lw=3 ; #de-correlation length cloud overlap method (Barker, 2008)
export iovr_sw=3 ; #de-correlation length cloud overlap method (Barker, 2008)
Expand Down
6 changes: 3 additions & 3 deletions parm/config/gfs/config.fcst
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ export esmf_logkind="ESMF_LOGKIND_MULTI_ON_ERROR" #Options: ESMF_LOGKIND_MULTI_O

export FORECASTSH="${SCRgfs}/exglobal_forecast.sh"
#export FORECASTSH="${SCRgfs}/exglobal_forecast.py" # Temp. while this is worked on
export FCSTEXEC="ufs_model.x"
export FCSTEXEC="gfs_model.x"

#######################################################################
# Model configuration
Expand Down Expand Up @@ -116,7 +116,7 @@ if (( gwd_opt == 2 )); then
export do_gsl_drag_ls_bl=".true."
export do_gsl_drag_ss=".false."
export do_gsl_drag_tofd=".true."
export do_gwd_opt_psl=".true."
export do_gwd_opt_psl=".true."
export do_ugwp_v1_orog_only=".false."
launch_level=$(echo "${LEVS}/2.35" |bc)
export launch_level
Expand Down Expand Up @@ -148,7 +148,7 @@ if [[ "${progsigma}" == ".true." ]]; then tbp="_progsigma" ; fi
if [[ "${DO_AERO_FCST}" == "YES" ]]; then
export IAER=2011 # spectral band mapping method for aerosol optical properties
else
export IAER=1011
export IAER=1011
fi
export iovr_lw=3 ; #de-correlation length cloud overlap method (Barker, 2008)
export iovr_sw=3 ; #de-correlation length cloud overlap method (Barker, 2008)
Expand Down
10 changes: 5 additions & 5 deletions scripts/exgfs_wave_post_pnt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -271,18 +271,18 @@ source "${USHgfs}/preamble.sh"
rm -f buoy_tmp.loc buoy_log.ww3 ww3_oup.inp
${NLN} ./out_pnt.${waveuoutpGRD} ./out_pnt.ww3
${NLN} ./mod_def.${waveuoutpGRD} ./mod_def.ww3
export pgm=ww3_outp;. prep_step
${EXECgfs}/ww3_outp > buoy_lst.loc 2>&1
export err=$?;err_chk

export pgm="${NET,,}_ww3_outp.x"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does SFS get its own executable or does it use the GEFS executable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually, it will. For now, SFS_POST=YES will use the gefs prefix.

. prep_step
aerorahul marked this conversation as resolved.
Show resolved Hide resolved

"${EXECgfs}/${pgm}" > buoy_lst.loc 2>&1
export err=$?;err_chk
if [ "$err" != '0' ] && [ ! -f buoy_log.ww3 ]
then
pgm=wave_post
set +x
echo ' '
echo '******************************************** '
echo '*** FATAL ERROR : ERROR IN ww3_outp *** '
echo "*** FATAL ERROR : ERROR IN ${pgm} *** "
echo '******************************************** '
echo ' '
cat buoy_tmp.loc
Expand Down
7 changes: 4 additions & 3 deletions sorc/build_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ _wave_opt=""
_hydro_opt=""
_build_job_max=20
_quick_kill="NO"
_ufs_exec="-e gfs_model.x"
# Reset option counter in case this script is sourced
OPTIND=1
while getopts ":a:dfghj:kuvwy" option; do
Expand All @@ -69,8 +70,8 @@ while getopts ":a:dfghj:kuvwy" option; do
k) _quick_kill="YES" ;;
u) _build_ufsda="YES" ;;
v) _verbose_opt="-v";;
w) _wave_opt="-w";;
y) _hydro_opt="-y";;
w) _wave_opt="-w"; _ufs_exec="-e gefs_model.x";;
y) _hydro_opt="-y"; _ufs_exec="-e sfs_model.x";;
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
:)
echo "[${BASH_SOURCE[0]}]: ${option} requires an argument"
_usage
Expand Down Expand Up @@ -133,7 +134,7 @@ declare -A build_opts
big_jobs=0
build_jobs["ufs"]=8
big_jobs=$((big_jobs+1))
build_opts["ufs"]="${_wave_opt} ${_hydro_opt} ${_verbose_opt} ${_build_ufs_opt} ${_build_debug}"
build_opts["ufs"]="${_wave_opt} ${_hydro_opt} ${_verbose_opt} ${_build_ufs_opt} ${_build_debug} ${_ufs_exec}"

build_jobs["upp"]=1
build_opts["upp"]="${_build_debug}"
Expand Down
21 changes: 15 additions & 6 deletions sorc/build_ufs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ APP="S2SWA"
CCPP_SUITES="FV3_GFS_v17_p8_ugwpv1,FV3_GFS_v17_coupled_p8_ugwpv1,FV3_global_nest_v1" # TODO: does the g-w need to build with all these CCPP_SUITES?
PDLIB="ON"
HYDRO="OFF"
EXEC_NAME="ufs_model.x"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under what circumstances should ufs_model.x be used? Since both config.fcsts reference gfs_model.x, gefs_model.x, or sfs_model.x, it may cause issues to have ufs_model.x as the default. I'd suggest at least a warning be raised if build_ufs.sh is called without specifying an executable to build.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can completely remove ufs_model.x if that is the source of confusion. ufs_model.x is going to be used if EXEC_NAME is not provided.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with a default, but I think it would make sense to have a default that is useable. How about defaulting to gfs_model.x?


while getopts ":da:fj:vwy" option; do
while getopts ":da:fj:e:vwy" option; do
case "${option}" in
d) BUILD_TYPE="Debug";;
a) APP="${OPTARG}";;
Expand All @@ -18,6 +19,7 @@ while getopts ":da:fj:vwy" option; do
v) export BUILD_VERBOSE="YES";;
w) PDLIB="OFF";;
y) HYDRO="ON";;
e) EXEC_NAME="${OPTARG}";;
:)
echo "[${BASH_SOURCE[0]}]: ${option} requires an argument"
;;
Expand All @@ -40,13 +42,20 @@ if [[ ${BUILD_TYPE:-"Release"} = "DEBUG" ]] ; then
elif [[ "${FASTER:-OFF}" == ON ]] ; then
MAKE_OPT+=" -DFASTER=ON"
fi
COMPILE_NR=0

case "${EXEC_NAME}" in
"ufs_model.x") COMPILE_ID=0 ;;
"gfs_model.x") COMPILE_ID=1 ;;
"gefs_model.x") COMPILE_ID=2 ;;
"sfs_model.x") COMPILE_ID=3 ;;
*) echo "Unsupported executable name: ${EXEC_NAME}"; exit 1 ;;
esac
CLEAN_BEFORE=YES
CLEAN_AFTER=NO

BUILD_JOBS=${BUILD_JOBS:-8} ./tests/compile.sh "${MACHINE_ID}" "${MAKE_OPT}" "${COMPILE_NR}" "intel" "${CLEAN_BEFORE}" "${CLEAN_AFTER}"
mv "./tests/fv3_${COMPILE_NR}.exe" ./tests/ufs_model.x
mv "./tests/modules.fv3_${COMPILE_NR}.lua" ./tests/modules.ufs_model.lua
cp "./modulefiles/ufs_common.lua" ./tests/ufs_common.lua
BUILD_JOBS=${BUILD_JOBS:-8} ./tests/compile.sh "${MACHINE_ID}" "${MAKE_OPT}" "${COMPILE_ID}" "intel" "${CLEAN_BEFORE}" "${CLEAN_AFTER}"
mv "./tests/fv3_${COMPILE_ID}.exe" "./tests/${EXEC_NAME}"
if [[ ! -f "./tests/modules.ufs_model.lua" ]]; then mv "./tests/modules.fv3_${COMPILE_ID}.lua" "./tests/modules.ufs_model.lua"; fi
if [[ ! -f "./tests/ufs_common.lua" ]]; then cp "./modulefiles/ufs_common.lua" ./tests/ufs_common.lua; fi
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved

exit 0
132 changes: 53 additions & 79 deletions sorc/build_ww3prepost.sh
Original file line number Diff line number Diff line change
@@ -1,131 +1,105 @@
#! /usr/bin/env bash
set -x

script_dir=$(dirname "${BASH_SOURCE[0]}")
cd "${script_dir}" || exit 1
# shellcheck disable=SC2312
_HOMEgfs=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )/.." && pwd -P)
Fixed Show fixed Hide fixed
cd "${_HOMEgfs}/sorc" || exit 1

# Default settings
APP="S2SWA"
PDLIB="ON"

while getopts ":j:a:dvw" option; do
case "${option}" in
a) APP="${OPTARG}";;
d) BUILD_TYPE="Debug";;
j) BUILD_JOBS="${OPTARG}";;
v) export BUILD_VERBOSE="YES";;
w) PDLIB="OFF";;
:)
echo "[${BASH_SOURCE[0]}]: ${option} requires an argument"
usage
;;
*)
echo "[${BASH_SOURCE[0]}]: Unrecognized option: ${option}"
usage
;;
d) BUILD_TYPE="Debug" ;;
j) BUILD_JOBS="${OPTARG}" ;;
v) export BUILD_VERBOSE="YES" ;;
w) PDLIB="OFF" ;;
:)
echo "[${BASH_SOURCE[0]}]: ${option} requires an argument"
usage
;;
*)
echo "[${BASH_SOURCE[0]}]: Unrecognized option: ${option}"
usage
;;
esac
done

# Determine which switch to use
if [[ "${APP}" == "ATMW" ]]; then
ww3switch="model/esmf/switch"
aerorahul marked this conversation as resolved.
Show resolved Hide resolved
else
if [[ "${PDLIB}" == "ON" ]]; then
ww3switch="model/bin/switch_meshcap_pdlib"
else
ww3switch="model/bin/switch_meshcap"
fi
fi

# Check final exec folder exists
if [[ ! -d "../exec" ]]; then
mkdir ../exec
fi

finalexecdir="$( pwd -P )/../exec"

#Determine machine and load modules
# Determine machine and load modules
set +x
source "${script_dir}/ufs_model.fd/tests/detect_machine.sh"
source "${script_dir}/ufs_model.fd/tests/module-setup.sh"
module use "${script_dir}/ufs_model.fd/modulefiles"
source "${_HOMEgfs}/sorc/ufs_model.fd/tests/detect_machine.sh"
source "${_HOMEgfs}/sorc/ufs_model.fd/tests/module-setup.sh"
module use "${_HOMEgfs}/sorc/ufs_model.fd/modulefiles"
module load "ufs_${MACHINE_ID}.intel"
set -x

#Set WW3 directory, switch, prep and post exes
cd ufs_model.fd/WW3 || exit 1
WW3_DIR=$( pwd -P )
#Set WW3 directory
cd "${_HOMEgfs}/sorc/ufs_model.fd/WW3" || exit 1
WW3_DIR=$(pwd -P)
export WW3_DIR
export SWITCHFILE="${WW3_DIR}/${ww3switch}"

# Build exes for prep jobs and post jobs:
prep_exes="ww3_grid ww3_prep ww3_prnc ww3_grid"
post_exes="ww3_outp ww3_outf ww3_outp ww3_gint ww3_ounf ww3_ounp ww3_grib"
# Determine which switch to use
if [[ "${PDLIB}" == "ON" ]]; then
ww3switch="model/bin/switch_meshcap_pdlib"
path_build="${WW3_DIR}/build/pdlib_ON"
path_install="${WW3_DIR}/install/pdlib_ON"
else
ww3switch="model/bin/switch_meshcap"
path_build="${WW3_DIR}/build/pdlib_OFF"
path_install="${WW3_DIR}/install/pdlib_OFF"
fi
export SWITCHFILE="${WW3_DIR}/${ww3switch}"

#create build directory:
path_build="${WW3_DIR}/build_SHRD"
[[ -d "${path_build}" ]] && rm -rf "${path_build}"
mkdir -p "${path_build}" || exit 1
cd "${path_build}" || exit 1
echo "Forcing a SHRD build"
aerorahul marked this conversation as resolved.
Show resolved Hide resolved

buildswitch="${path_build}/switch"

cat "${SWITCHFILE}" > "${path_build}/tempswitch"

sed -e "s/DIST/SHRD/g"\
-e "s/OMPG / /g"\
-e "s/OMPH / /g"\
-e "s/MPIT / /g"\
-e "s/MPI / /g"\
-e "s/B4B / /g"\
-e "s/PDLIB / /g"\
-e "s/SCOTCH / /g"\
-e "s/METIS / /g"\
-e "s/NOGRB/NCEP2/g"\
"${path_build}/tempswitch" > "${path_build}/switch"
cat "${SWITCHFILE}" >"${path_build}/tempswitch"

sed -e "s/DIST/SHRD/g" \
-e "s/OMPG / /g" \
-e "s/OMPH / /g" \
-e "s/MPIT / /g" \
-e "s/MPI / /g" \
-e "s/B4B / /g" \
-e "s/PDLIB / /g" \
-e "s/SCOTCH / /g" \
-e "s/METIS / /g" \
-e "s/NOGRB/NCEP2/g" \
"${path_build}/tempswitch" >"${path_build}/switch"
rm "${path_build}/tempswitch"

echo "Switch file is ${buildswitch} with switches:"
cat "${buildswitch}"

#define cmake build options
MAKE_OPT="-DCMAKE_INSTALL_PREFIX=install"
MAKE_OPT="-DCMAKE_INSTALL_PREFIX=${path_install}"
[[ ${BUILD_TYPE:-"Release"} = "Debug" ]] && MAKE_OPT+=" -DCMAKE_BUILD_TYPE=Debug"

#Build executables:
# shellcheck disable=SC2086
cmake "${WW3_DIR}" -DSWITCH="${buildswitch}" ${MAKE_OPT}
rc=$?
if (( rc != 0 )); then
if ((rc != 0)); then
echo "Fatal error in cmake."
exit "${rc}"
fi

make -j "${BUILD_JOBS:-8}"
rc=$?
if (( rc != 0 )); then
if ((rc != 0)); then
echo "Fatal error in make."
exit "${rc}"
fi

make install
if (( rc != 0 )); then
rc=$?
if ((rc != 0)); then
echo "Fatal error in make install."
exit "${rc}"
fi

# Copy to top-level exe directory
for prog in ${prep_exes} ${post_exes}; do
cp "${path_build}/install/bin/${prog}" "${finalexecdir}/"
JessicaMeixner-NOAA marked this conversation as resolved.
Show resolved Hide resolved
rc=$?
if (( rc != 0 )); then
echo "FATAL: Unable to copy ${path_build}/${prog} to ${finalexecdir} (Error code ${rc})"
exit "${rc}"
fi
done

#clean-up build directory:
echo "executables are in ${finalexecdir}"
echo "cleaning up ${path_build}"
rm -rf "${path_build}"

exit 0
Loading