diff --git a/README.md b/README.md index 067b2a16b2..0aedf1e919 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ https://epic.noaa.gov/wp-content/uploads/2022/12/Debugging-Guide.pdf The SRW App v2.2.0 citation is as follows and should be used when presenting results based on research conducted with the App: -UFS Development Team. (2023, Oct. 30). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v2.2.0). Zenodo. https://doi.org/10.5281/zenodo.10015544 +UFS Development Team. (2023, Oct. 31). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v2.2.0). Zenodo. https://doi.org/10.5281/zenodo.10015544 [![Python unittests](https://github.com/ufs-community/ufs-srweather-app/actions/workflows/python_unittests.yaml/badge.svg)](https://github.com/ufs-community/ufs-srweather-app/actions/workflows/python_unittests.yaml) [![Python functional tests](https://github.com/ufs-community/ufs-srweather-app/actions/workflows/python_func_tests.yaml/badge.svg)](https://github.com/ufs-community/ufs-srweather-app/actions/workflows/python_func_tests.yaml) diff --git a/devbuild.sh b/devbuild.sh index b21998fc8e..b0c5c9a7d7 100755 --- a/devbuild.sh +++ b/devbuild.sh @@ -417,6 +417,19 @@ else fi module list +# Apply patch for sorc/CMakeLists.txt for MacOS arm64/aarch64 +if [[ "${PLATFORM}" == "macos" ]]; then + ARCH=$(uname -m) + if [[ "${ARCH}" == arm64 ]] || [[ "${ARCH}" == aarch64 ]]; then + patch1="sorc/patch_macos_arm64_sorc_cmakelists.txt" + if patch -p1 -R --dry-run --silent -d ./sorc -N < ${patch1} 1> /dev/null; then + echo "Patch ${patch1} was already applied"; + else + patch -p1 -d ./sorc -N < ${patch1} + fi + fi +fi + mkdir -p ${BUILD_DIR} cd ${BUILD_DIR} diff --git a/docs/UsersGuide/source/BackgroundInfo/Introduction.rst b/docs/UsersGuide/source/BackgroundInfo/Introduction.rst index 926dfd61b8..639df3dfd9 100644 --- a/docs/UsersGuide/source/BackgroundInfo/Introduction.rst +++ b/docs/UsersGuide/source/BackgroundInfo/Introduction.rst @@ -31,7 +31,7 @@ Since the last release, developers have added a variety of features: The SRW App v2.2.0 citation is as follows and should be used when presenting results based on research conducted with the App: -UFS Development Team. (2023, Oct. 30). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v2.2.0). Zenodo. https://doi.org/10.5281/zenodo.10015544 +UFS Development Team. (2023, Oct. 31). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v2.2.0). Zenodo. https://doi.org/10.5281/zenodo.10015544 User's Guide Organization ============================ diff --git a/docs/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst b/docs/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst index a5d58985c1..164ec81d41 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst @@ -15,7 +15,7 @@ The Unified Forecast System (:term:`UFS`) Short-Range Weather (SRW) Application To build the SRW App, users will complete the following steps: - #. :ref:`Install prerequisites ` + #. :ref:`Install prerequisites ` #. :ref:`Clone the SRW App from GitHub ` #. :ref:`Check out the external repositories ` #. :ref:`Set up the build environment and build the executables ` @@ -358,9 +358,8 @@ Additional Details for Building on MacOS or Generic Linux The SRW App can be built on MacOS and generic Linux machines after the prerequisite software has been installed on these systems (via :term:`HPC-Stack` or :term:`spack-stack`). The installation for MacOS is architecture-independent and has been tested using both x86_64 and M1 chips (running natively). The following configurations for MacOS have been tested: - #. MacBookPro 2019, 2.4 GHz 8-core Intel Core i9 (x86_64), Monterey Sur 12.1, GNU compiler suite v.11.3.0 (gcc, gfortran, g++); mpich 3.3.2 or openmpi/4.1.2 - #. MacBookAir 2020, M1 chip (arm64, running natively), 4+4 cores, Big Sur 11.6.4, GNU compiler suite v.11.3.0 (gcc, gfortran, g++); mpich 3.3.2 or openmpi/4.1.2 - #. MacBook Pro 2015, 2.8 GHz Quad-Core Intel Core i7 (x86_64), Catalina OS X 10.15.7, GNU compiler suite v.11.2.0_3 (gcc, gfortran, g++); mpich 3.3.2 or openmpi/4.1.2 + #. MacBookPro 2019, 2.4 GHz 8-core Intel Core i9 (x86_64), OS Monterey 12.6.1, 32 GB RAM; GNU compiler suite v.12.3.0 (gcc, gfortran, g++); openmpi/4.1.5 + #. MacBookAir 2020, M1 chip (arm64, running natively), 4+4 cores, OS Ventura 13.0.1, 16 GB RAM; GNU compiler suite v.12.3.0 (gcc, gfortran, g++); openmpi/4.1.5 Several Linux builds have been tested on systems with x86_64 architectures. diff --git a/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst b/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst index 5e0f2db31b..6d17dab6e7 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst @@ -80,9 +80,9 @@ On most Level 1 systems, a container named ``ubuntu20.04-intel-srwapp-release-pu * - Machine - File Location - * - Cheyenne/Derecho + * - Cheyenne/Derecho `*`_ - /glade/scratch/epicufsrt/containers - * - Gaea + * - Gaea `*`_ - /lustre/f2/dev/role.epic/containers * - Hera - /scratch1/NCEPDEV/nems/role.epic/containers @@ -90,9 +90,13 @@ On most Level 1 systems, a container named ``ubuntu20.04-intel-srwapp-release-pu - /mnt/lfs4/HFIP/hfv3gfs/role.epic/containers * - NOAA Cloud - /contrib/EPIC/containers - * - Orion/Hercules + * - Orion/Hercules `*`_ - /work/noaa/epic/role-epic/contrib/containers +.. _`*` : + + \* On these systems, container testing shows inconsistent results. + .. note:: * On Gaea, Singularity/Apptainer is only available on the C5 partition, and therefore container use is only supported on Gaea C5. * The NOAA Cloud containers are accessible only to those with EPIC resources. diff --git a/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst b/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst index 9d2b288109..a0065dc553 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst @@ -192,29 +192,100 @@ MacOS requires the installation of a few additional packages and, possibly, an u Creating the |wflow_env| Environment on Linux and Mac OS """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -On generic Mac and Linux systems, users need to create a conda |wflow_env| environment. The environment can be stored in a local path, which could be a default location or a user-specified location (e.g., ``$HOME/condaenv/venvs/`` directory). (To determine the default location, use the ``conda info`` command, and look for the ``envs directories`` list.) The following is a brief recipe for creating a virtual conda environment on non-Level 1 platforms. It uses the aarch64 (64-bit ARM) Miniforge for Linux and installs into $HOME/conda. Adjust as necessary for your target system. +On generic Mac and Linux systems, users need to create a conda |wflow_env| environment that contains python packages required for running the workflow. Other conda environments may need to be activated for running graphics generation tasks (|graphics_env|) or when testing the AQM/CMAQ (|cmaq_env|). Python packages in these other environments may conflict with those in |wflow_env|. The environments can be stored in a local path, which can be a default location or a user-specified location (e.g., ``$HOME/condaenv/venvs/`` directory). (To determine the default location, use the ``conda info`` command, and look for the ``envs directories`` list.) +These conda environments can be added to the existing python or conda modules. -.. code-block:: console +There are several options available for building virtual conda environments on non-Level 1 platforms. The examples in this section use the aarch64 (64-bit ARM) Miniforge for Linux and install into ``$HOME/conda``. Users should adjust as needed for their target system. - wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh - bash Miniforge3-Linux-aarch64.sh -bfp ~/conda - rm Miniforge3-Linux-aarch64.sh - source ~/conda/etc/profile.d/conda.sh - conda activate - conda install -y conda-build conda-verify - cd path/to/your/workflow-tools/clone - conda build recipe - conda create -y -n workflow_tools -c local workflow_tools - conda activate workflow_tools +**Options:** -In future shells, you can activate and use this environment with: -.. code-block:: console +1) Users can add the following environment .yaml files: + + a) workflow_tools.yaml for |wflow_env| + + .. code-block:: console + + name: workflow_tools + channels: + - conda-forge + - defaults + dependencies: + - python=3.9* + - boto3=1.22* + - black + - f90nml=1.4* + - jinja2=3.0* + - numpy=1.21* + - pylint + - pytest + - pyyaml=6.0* + - tox + + b) regional_workflow.yaml for |graphics_env| + + .. code-block:: console + + name: regional_workflow + channels: + - conda-forge + - defaults + dependencies: + - python=3.9.* + - f90nml + - jinja2 + - pyyaml + - scipy + - matplotlib=3.5.2* + - pygrib + - cartopy + + c) regional_workflow_cmaq.yaml for |cmaq_env| + + .. code-block:: console + + name: regional_workflow_cmaq + channels: + - conda-forge + - defaults + dependencies: + - python=3.9.12 + - f90nml=1.4* + - jinja2=3.0* + - pyyaml=6.0* + - scipy + - matplotlib + - pygrib + - cartopy + - netcdf4 + - xarray + +2) Users can instead install Miniforge. This example uses the aarch64 (64-bit ARM) Miniforge for Linux that installs into ``$HOME/conda``. Users should adjust as needed for their target system. + + .. code-block:: console + + wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh + bash Miniforge3-Linux-aarch64.sh -bfp ~/conda + rm Miniforge3-Linux-aarch64.sh + source ~/conda/etc/profile.d/conda.sh + conda activate + conda install -y conda-build conda-verify + cd path/to/your/workflow-tools/clone + conda build recipe + conda create -y -n workflow_tools -c local workflow_tools + conda activate workflow_tools + + In future shells, you can activate and use this environment with: + + .. code-block:: console + + source ~/conda/etc/profile.d/conda.sh + conda activate workflow_tools + + See the `workflow-tools repository `__ for additional documentation. - source ~/conda/etc/profile.d/conda.sh - conda activate workflow_tools +3) A third option is to build miniconda3 and create an Lmod modulefile that can be loaded with other modules during the workflow. The module can be added to the user's ``wflow_.lua`` modulefile, and the environments can be activated or deactivated as needed for a particular workflow task. A repository with full installation instructions, a modulefile template, and environment configuration files can be accessed in `NOAA-EPIC/miniconda3 repository `__. Full instructions can be viewed in the `README.md file `__. -See the `workflow-tools repository `__ for additional documentation. Modify a ``wflow_`` File `````````````````````````````````````` @@ -632,7 +703,7 @@ This can be helpful when conducting multiple experiments with different types of Plotting Configuration (optional) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -An optional Python plotting task (PLOT_ALLVARS) can be activated in the workflow to generate plots for the :term:`FV3`-:term:`LAM` post-processed :term:`GRIB2` +An optional Python plotting task (plot_allvars) can be activated in the workflow to generate plots for the :term:`FV3`-:term:`LAM` post-processed :term:`GRIB2` output over the :term:`CONUS`. It generates graphics plots for a number of variables, including: * 2-m temperature diff --git a/docs/UsersGuide/source/CustomizingTheWorkflow/InputOutputFiles.rst b/docs/UsersGuide/source/CustomizingTheWorkflow/InputOutputFiles.rst index 416daa91b0..6312aabf7e 100644 --- a/docs/UsersGuide/source/CustomizingTheWorkflow/InputOutputFiles.rst +++ b/docs/UsersGuide/source/CustomizingTheWorkflow/InputOutputFiles.rst @@ -164,8 +164,8 @@ If users wish to modify the fields or levels that are output from the UPP, they This process requires advanced knowledge of which fields can be output for the UFS Weather Model. UPP Product Output Tables for the UFS SRW LAM Grid: - * :doc:`3D Native Hybrid Level Fields ` - * :doc:`3D Pressure Level Fields ` + * :ref:`3D Native Hybrid Level Fields ` + * :ref:`3D Pressure Level Fields ` Use the instructions in the `UPP User's Guide `__ to make modifications to the ``fv3lam.xml`` file and to remake the flat text file, called ``postxconfig-NT-fv3lam.txt`` (default), that the UPP reads. diff --git a/docs/UsersGuide/source/conf.py b/docs/UsersGuide/source/conf.py index a581d0e410..ca6a4bfe79 100644 --- a/docs/UsersGuide/source/conf.py +++ b/docs/UsersGuide/source/conf.py @@ -89,6 +89,8 @@ rst_prolog = """ .. |wflow_env| replace:: ``workflow_tools`` +.. |graphics_env| replace:: ``regional_workflow`` +.. |cmaq_env| replace:: ``regional_workflow_cmaq`` .. |activate| replace:: ``conda activate workflow_tools`` .. |prompt| replace:: ``(workflow_tools)`` .. |latestr| replace:: v2.2.0 diff --git a/docs/UsersGuide/source/tables/SRW_NATLEV_table.rst b/docs/UsersGuide/source/tables/SRW_NATLEV_table.rst index 75cbc71789..50a2f4b76a 100644 --- a/docs/UsersGuide/source/tables/SRW_NATLEV_table.rst +++ b/docs/UsersGuide/source/tables/SRW_NATLEV_table.rst @@ -1,5 +1,7 @@ :orphan: +.. _SRW_NATLEV_table: + ************************************************************ Fields Requested in the UPP Parameter Table for SRW NATLEV ************************************************************ diff --git a/docs/UsersGuide/source/tables/SRW_PRSLEV_table.rst b/docs/UsersGuide/source/tables/SRW_PRSLEV_table.rst index 57fd8cc69b..da5226d375 100644 --- a/docs/UsersGuide/source/tables/SRW_PRSLEV_table.rst +++ b/docs/UsersGuide/source/tables/SRW_PRSLEV_table.rst @@ -1,5 +1,7 @@ :orphan: +.. _SRW_PRSLEV_table: + ********************************************************** Fields Requested in the UPP Parameter Table for SRW PRSLEV ********************************************************** diff --git a/etc/lmod-setup.csh b/etc/lmod-setup.csh index dc8fb2b064..4369725bf4 100644 --- a/etc/lmod-setup.csh +++ b/etc/lmod-setup.csh @@ -18,22 +18,23 @@ if ( "$L_MACHINE" != wcoss2 ) then endif if ( "$L_MACHINE" == macos ) then - arch=$(uname -m) - [[ "$arch" = arm64 ]] && export ENV="/opt/homebrew/opt/lmod/init/csh" - [[ "$arch" = x86_64 ]] && export ENV="/usr/local/opt/lmod/init/csh" - source $ENV + if ( -d /opt/homebrew/opt/lmod ) then + source /opt/homebrew/opt/lmod/init/csh + else if ( -d /usr/local/opt/lmod ) then + source /usr/local/opt/lmod/init/csh + else + echo "Path for Lmod is unknown, verify whether Lmod is installed" + endif module purge else if ( "$L_MACHINE" == linux ) then - setenv ENV "/usr/share/lmod/lmod/init/csh" - source $ENV + source /usr/share/lmod/lmod/init/csh module purge else if ( "$L_MACHINE" == singularity ) then - set ENV="/usr/share/lmod/lmod/init/csh" - source $ENV + source /usr/share/lmod/lmod/init/csh module purge diff --git a/etc/lmod-setup.sh b/etc/lmod-setup.sh index 8a6b651958..c654e631a0 100644 --- a/etc/lmod-setup.sh +++ b/etc/lmod-setup.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash if [ $# = 0 ]; then L_MACHINE=${MACHINE} @@ -26,10 +26,12 @@ fi if [ "$L_MACHINE" = macos ]; then arch=$(uname -m) - [[ "$arch" = arm64 ]] && export BASH_ENV="/opt/homebrew/opt/lmod/init/bash" - [[ "$arch" = x86_64 ]] && export BASH_ENV="/usr/local/opt/lmod/init/bash" + if [ "$arch" = arm64 ] || [ "$arch" = aarch64 ]; then + export BASH_ENV="/opt/homebrew/opt/lmod/init/bash" + else + export BASH_ENV="/usr/local/opt/lmod/init/bash" + fi source $BASH_ENV - module purge elif [ "$L_MACHINE" = linux ]; then diff --git a/jobs/JREGIONAL_AQM_ICS b/jobs/JREGIONAL_AQM_ICS index ce3e539cc9..5c8ba9c8dd 100755 --- a/jobs/JREGIONAL_AQM_ICS +++ b/jobs/JREGIONAL_AQM_ICS @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_AQM_LBCS b/jobs/JREGIONAL_AQM_LBCS index 7f13d12fa1..c711f90288 100755 --- a/jobs/JREGIONAL_AQM_LBCS +++ b/jobs/JREGIONAL_AQM_LBCS @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_BIAS_CORRECTION_O3 b/jobs/JREGIONAL_BIAS_CORRECTION_O3 index 6586d4427c..ddcef59494 100755 --- a/jobs/JREGIONAL_BIAS_CORRECTION_O3 +++ b/jobs/JREGIONAL_BIAS_CORRECTION_O3 @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_BIAS_CORRECTION_PM25 b/jobs/JREGIONAL_BIAS_CORRECTION_PM25 index a0c0ba1b4a..7e08b02a12 100755 --- a/jobs/JREGIONAL_BIAS_CORRECTION_PM25 +++ b/jobs/JREGIONAL_BIAS_CORRECTION_PM25 @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_CHECK_POST_OUTPUT b/jobs/JREGIONAL_CHECK_POST_OUTPUT index b9c1868282..2b1fe69bbb 100755 --- a/jobs/JREGIONAL_CHECK_POST_OUTPUT +++ b/jobs/JREGIONAL_CHECK_POST_OUTPUT @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_FIRE_EMISSION b/jobs/JREGIONAL_FIRE_EMISSION index ee540cfa6e..fdb6e57b0a 100755 --- a/jobs/JREGIONAL_FIRE_EMISSION +++ b/jobs/JREGIONAL_FIRE_EMISSION @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_GET_EXTRN_MDL_FILES b/jobs/JREGIONAL_GET_EXTRN_MDL_FILES index 6d8dd23c7e..8efd332dd9 100755 --- a/jobs/JREGIONAL_GET_EXTRN_MDL_FILES +++ b/jobs/JREGIONAL_GET_EXTRN_MDL_FILES @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_MAKE_GRID b/jobs/JREGIONAL_MAKE_GRID index 0a6adfe277..844d782bc7 100755 --- a/jobs/JREGIONAL_MAKE_GRID +++ b/jobs/JREGIONAL_MAKE_GRID @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_MAKE_ICS b/jobs/JREGIONAL_MAKE_ICS index 5ff0b3e4d8..1e38f4058d 100755 --- a/jobs/JREGIONAL_MAKE_ICS +++ b/jobs/JREGIONAL_MAKE_ICS @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_MAKE_LBCS b/jobs/JREGIONAL_MAKE_LBCS index 0070654cbe..4c524e26a6 100755 --- a/jobs/JREGIONAL_MAKE_LBCS +++ b/jobs/JREGIONAL_MAKE_LBCS @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_MAKE_OROG b/jobs/JREGIONAL_MAKE_OROG index 8f50c134ff..b6f674e5ee 100755 --- a/jobs/JREGIONAL_MAKE_OROG +++ b/jobs/JREGIONAL_MAKE_OROG @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_MAKE_SFC_CLIMO b/jobs/JREGIONAL_MAKE_SFC_CLIMO index fe7773230c..eee25b193a 100755 --- a/jobs/JREGIONAL_MAKE_SFC_CLIMO +++ b/jobs/JREGIONAL_MAKE_SFC_CLIMO @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_NEXUS_EMISSION b/jobs/JREGIONAL_NEXUS_EMISSION index 0f1a00d1a6..915de0f054 100755 --- a/jobs/JREGIONAL_NEXUS_EMISSION +++ b/jobs/JREGIONAL_NEXUS_EMISSION @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_NEXUS_GFS_SFC b/jobs/JREGIONAL_NEXUS_GFS_SFC index 5930762106..5fc05f86db 100755 --- a/jobs/JREGIONAL_NEXUS_GFS_SFC +++ b/jobs/JREGIONAL_NEXUS_GFS_SFC @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_NEXUS_POST_SPLIT b/jobs/JREGIONAL_NEXUS_POST_SPLIT index fc21421f8d..7cb8a55bf0 100755 --- a/jobs/JREGIONAL_NEXUS_POST_SPLIT +++ b/jobs/JREGIONAL_NEXUS_POST_SPLIT @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_PLOT_ALLVARS b/jobs/JREGIONAL_PLOT_ALLVARS index 49eb422c3a..5e59abd93d 100755 --- a/jobs/JREGIONAL_PLOT_ALLVARS +++ b/jobs/JREGIONAL_PLOT_ALLVARS @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_POINT_SOURCE b/jobs/JREGIONAL_POINT_SOURCE index d3a02b7534..57000dd599 100755 --- a/jobs/JREGIONAL_POINT_SOURCE +++ b/jobs/JREGIONAL_POINT_SOURCE @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_POST_STAT_O3 b/jobs/JREGIONAL_POST_STAT_O3 index af8652e05d..a522d00dbb 100755 --- a/jobs/JREGIONAL_POST_STAT_O3 +++ b/jobs/JREGIONAL_POST_STAT_O3 @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_POST_STAT_PM25 b/jobs/JREGIONAL_POST_STAT_PM25 index 81ec818075..cd86879a73 100755 --- a/jobs/JREGIONAL_POST_STAT_PM25 +++ b/jobs/JREGIONAL_POST_STAT_PM25 @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_PRE_POST_STAT b/jobs/JREGIONAL_PRE_POST_STAT index 41cb25d6a0..640c629bce 100755 --- a/jobs/JREGIONAL_PRE_POST_STAT +++ b/jobs/JREGIONAL_PRE_POST_STAT @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_FCST b/jobs/JREGIONAL_RUN_FCST index 383739e172..a7f7c96031 100755 --- a/jobs/JREGIONAL_RUN_FCST +++ b/jobs/JREGIONAL_RUN_FCST @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_MET_GENENSPROD_OR_ENSEMBLESTAT b/jobs/JREGIONAL_RUN_MET_GENENSPROD_OR_ENSEMBLESTAT index 0f70f0f285..707697b5ab 100755 --- a/jobs/JREGIONAL_RUN_MET_GENENSPROD_OR_ENSEMBLESTAT +++ b/jobs/JREGIONAL_RUN_MET_GENENSPROD_OR_ENSEMBLESTAT @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX b/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX index a9f7d3ae98..0301e9946a 100755 --- a/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX +++ b/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSMEAN b/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSMEAN index 249a0a81bd..ab08320f33 100755 --- a/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSMEAN +++ b/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSMEAN @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSPROB b/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSPROB index 7239b7367c..7da98212ac 100755 --- a/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSPROB +++ b/jobs/JREGIONAL_RUN_MET_GRIDSTAT_OR_POINTSTAT_VX_ENSPROB @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_MET_PB2NC_OBS b/jobs/JREGIONAL_RUN_MET_PB2NC_OBS index 552d855214..e36e72418f 100755 --- a/jobs/JREGIONAL_RUN_MET_PB2NC_OBS +++ b/jobs/JREGIONAL_RUN_MET_PB2NC_OBS @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_MET_PCPCOMBINE b/jobs/JREGIONAL_RUN_MET_PCPCOMBINE index 2722a8e646..7364ed96c9 100755 --- a/jobs/JREGIONAL_RUN_MET_PCPCOMBINE +++ b/jobs/JREGIONAL_RUN_MET_PCPCOMBINE @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_POST b/jobs/JREGIONAL_RUN_POST index 459431522f..b4327667a0 100755 --- a/jobs/JREGIONAL_RUN_POST +++ b/jobs/JREGIONAL_RUN_POST @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/jobs/JREGIONAL_RUN_PRDGEN b/jobs/JREGIONAL_RUN_PRDGEN index 6621db1727..2d30ced9db 100755 --- a/jobs/JREGIONAL_RUN_PRDGEN +++ b/jobs/JREGIONAL_RUN_PRDGEN @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/modulefiles/build_macos_gnu.lua b/modulefiles/build_macos_gnu.lua index 266bfd11a6..cbef28c74f 100644 --- a/modulefiles/build_macos_gnu.lua +++ b/modulefiles/build_macos_gnu.lua @@ -9,7 +9,7 @@ if mode() == "load" then execute{cmd="ulimit -S -s unlimited", modeA={"load"}} end --- This path should point to your HPCstack installation directory +-- This path points to your HPCstack installation's base directory local HPCstack="/Users/username/hpc-stack/install" -- Load HPC stack @@ -21,11 +21,19 @@ load("hpc-gnu") load("hpc-openmpi") load("srw_common") +load("nccmp") +load("nco") --- MacOS with arm64 architecture: `uname -m` expands to arm64 --- MacOS with Intel architecture: `uname -m` expands to x86_64 -local arch = 'eval $(uname -m)' -if (arch == "arm64") then +function file_exists(name) + local f=io.open(name,"r") + if f~=nil then io.close(f) return true else return false end +end +-- Check location of Gnu compilers installed by homebrew +-- M1/M2 arm64/aarch64: /opt/homebrew/bin +-- Intel x86_64: /usr/local/bin +local arm64=file_exists("/opt/homebrew/bin/gcc") + +if arm64 then setenv("CC", "/opt/homebrew/bin/gcc") setenv("FC", "/opt/homebrew/bin/gfortran") setenv("CXX", "/opt/homebrew/bin/g++") @@ -34,29 +42,28 @@ else setenv("FC", "/usr/local/bin/gfortran") setenv("CXX", "/usr/local/bin/g++") end - -- Set MPI compilers depending on the MPI libraries built: local MPI_CC="mpicc" local MPI_CXX="mpicxx" local MPI_FC="mpif90" - -- Set compilers and platform names for CMake: -setenv("CMAKE_C_COMPILER", MPI_CC) -setenv("CMAKE_CXX_COMPILER", MPI_CXX) -setenv("CMAKE_Fortran_COMPILER", MPI_FC) +setenv("CMAKE_C_COMPILER", "gcc") +setenv("CMAKE_CXX_COMPILER", "g++") +setenv("CMAKE_Fortran_COMPILER", "gfortran") -setenv("CMAKE_Platform", "macos.gnu") ---setenv("CMAKE_Platform", "macos.intel") +setenv("CMAKE_MPI_Fortran_COMPILER", MPI_FC) +setenv("CMAKE_MPI_C_COMPILER", MPI_CC) +setenv("CMAKE_MPI_CXX_COMPILER", MPI_CXX) -setenv("CMAKE_Fortran_COMPILER_ID", "GNU") ---setenv("CMAKE_Fortran_COMPILER_ID", "Intel") +-- Set compiler and linker flags if needed: +setenv("FFLAGS", " -fallow-argument-mismatch -fallow-invalid-boz -march=native ") +setenv("CFLAGS", " -march=native") --- Set compiler and linker flags if needed: -setenv("FFLAGS", " -DNO_QUAD_PRECISION -fallow-argument-mismatch ") +setenv("CMAKE_Platform", "macos.gnu") -- export the env. variable LDFLAGS after loading the current module --- export LDFLAGS="-L$MPI_ROOT/lib" +-- export LDFLAGS+=" -L$MPI_ROOT/lib " if mode() == "load" then LmodMsgRaw([===[ Please export env. variable LDFLAGS after the module is successfully loaded: @@ -64,4 +71,3 @@ if mode() == "load" then ]===]) end - diff --git a/modulefiles/srw_common.lua b/modulefiles/srw_common.lua index 6cd326a1f8..345cbd4357 100644 --- a/modulefiles/srw_common.lua +++ b/modulefiles/srw_common.lua @@ -1,5 +1,5 @@ -load_any("jasper/2.0.25","jasper/2.0.32") -load_any("zlib/1.2.11","zlib/1.2.13") +load_any("jasper") +load_any("zlib") load_any("png/1.6.37","libpng/1.6.37") load_any("netcdf/4.9.2","netcdf-c/4.9.2") @@ -20,10 +20,10 @@ load_any("gftl-shared/v1.5.0","gftl-shared/1.5.0") load_any("yafyaml/v0.5.1","yafyaml/0.5.1") load("mapl/2.35.2-esmf-8.4.2") -load_any("nemsio/2.5.2","nemsio/2.5.4") +load_any("nemsio") load("sfcio/1.4.1") load("sigio/2.3.2") load("w3nco/2.4.1") load_any("wrf_io/1.2.0","wrf-io/1.2.0") -load_any("wgrib2/2.0.8","wgrib2/3.1.1") +load_any("wgrib2") diff --git a/modulefiles/tasks/macos/plot_allvars.local.lua b/modulefiles/tasks/macos/plot_allvars.local.lua new file mode 100644 index 0000000000..22cfbcd4b5 --- /dev/null +++ b/modulefiles/tasks/macos/plot_allvars.local.lua @@ -0,0 +1,4 @@ +prepend_path("MODULEPATH","/Users/username/miniconda3/modulefiles") +load(pathJoin("miniconda3", os.getenv("miniconda3_ver") or "23.9.0")) + +setenv("SRW_ENV", "regional_workflow") diff --git a/modulefiles/tasks/macos/python_srw.lua b/modulefiles/tasks/macos/python_srw.lua new file mode 100644 index 0000000000..a07b577031 --- /dev/null +++ b/modulefiles/tasks/macos/python_srw.lua @@ -0,0 +1,4 @@ +prepend_path("MODULEPATH","/Users/username/miniconda3/modulefiles") +load(pathJoin("miniconda3", os.getenv("miniconda3_ver") or "23.9.0")) + +setenv("SRW_ENV", "workflow_tools") diff --git a/modulefiles/tasks/macos/run_vx.local.lua b/modulefiles/tasks/macos/run_vx.local.lua new file mode 100644 index 0000000000..54cc632c21 --- /dev/null +++ b/modulefiles/tasks/macos/run_vx.local.lua @@ -0,0 +1,25 @@ +--[[ +Compiler-specific modules are used for met and metplus libraries +--]] +local met_ver = (os.getenv("met_ver") or "10.1.2") +local metplus_ver = (os.getenv("metplus_ver") or "4.1.3") +if (mode() == "load") then + load(pathJoin("met", met_ver)) + load(pathJoin("metplus",metplus_ver)) +end +local base_met = os.getenv("met_ROOT") or os.getenv("MET_ROOT") +local base_metplus = os.getenv("metplus_ROOT") or os.getenv("METPLUS_ROOT") + +setenv("MET_INSTALL_DIR", base_met) +setenv("MET_BIN_EXEC", pathJoin(base_met,"bin")) +setenv("MET_BASE", pathJoin(base_met,"share/met")) +setenv("MET_VERSION", met_ver) +setenv("METPLUS_VERSION", metplus_ver) +setenv("METPLUS_ROOT", base_metplus) +setenv("METPLUS_PATH", base_metplus) + +if (mode() == "unload") then + unload(pathJoin("met", met_ver)) + unload(pathJoin("metplus",metplus_ver)) +end +load("python_srw") diff --git a/modulefiles/wflow_macos.lua b/modulefiles/wflow_macos.lua index 6ee6022b20..b904620980 100644 --- a/modulefiles/wflow_macos.lua +++ b/modulefiles/wflow_macos.lua @@ -1,35 +1,13 @@ help([[ -This module set a path needed to activate conda environement for running UFS SRW App on general macOS +This module set a path needed to activate conda environement for running UFS SRW App on general macOS, following miniconda3 module and conda environments installations ]]) whatis([===[This module activates conda environment for running the UFS SRW App on macOS]===]) setenv("CMAKE_Platform", "macos") --- Conda initialization function -function init_conda(conda_path) - local shell=myShellType() - local conda_file - if shell == "csh" then - conda_file=pathJoin(conda_path,"etc/profile.d/conda.csh") - else - conda_file=pathJoin(conda_path,"etc/profile.d/conda.sh") - end - local mcmd="source " .. conda_file - execute{cmd=mcmd, modeA={"load"}} -end - --- initialize conda -local conda_path="/Users/username/miniconda3" -init_conda(conda_path) - --- add rocoto to path -local rocoto_path="/Users/username/rocoto" -prepend_path("PATH", pathJoin(rocoto_path,"bin")) - --- add fake slurm commands -local srw_path="/Users/username/ufs-srweather-app" -prepend_path("PATH", pathJoin(srw_path, "ush/rocoto_fake_slurm")) +prepend_path("MODULEPATH","/Users/username/miniconda3/modulefiles") +load(pathJoin("miniconda3", os.getenv("miniconda3_ver") or "23.9.0")) -- set python path load("set_pythonpath") @@ -37,7 +15,7 @@ load("set_pythonpath") -- display conda activation message if mode() == "load" then LmodMsgRaw([===[Please do the following to activate conda virtual environment: - > conda activate workflow_tools" + > conda activate workflow_tools ]===]) end diff --git a/scripts/exregional_aqm_ics.sh b/scripts/exregional_aqm_ics.sh index c59bf87f81..676cc4ed90 100755 --- a/scripts/exregional_aqm_ics.sh +++ b/scripts/exregional_aqm_ics.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_aqm_lbcs.sh b/scripts/exregional_aqm_lbcs.sh index 24bac24ad7..09a33d40a2 100755 --- a/scripts/exregional_aqm_lbcs.sh +++ b/scripts/exregional_aqm_lbcs.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_bias_correction_o3.sh b/scripts/exregional_bias_correction_o3.sh index 79ed056f59..709cc1957d 100755 --- a/scripts/exregional_bias_correction_o3.sh +++ b/scripts/exregional_bias_correction_o3.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_bias_correction_pm25.sh b/scripts/exregional_bias_correction_pm25.sh index 05348ecbc8..9503f744c9 100755 --- a/scripts/exregional_bias_correction_pm25.sh +++ b/scripts/exregional_bias_correction_pm25.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_check_post_output.sh b/scripts/exregional_check_post_output.sh index d088e218cd..bab7599eb4 100755 --- a/scripts/exregional_check_post_output.sh +++ b/scripts/exregional_check_post_output.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_fire_emission.sh b/scripts/exregional_fire_emission.sh index a2e9f9b44d..e1d2101477 100755 --- a/scripts/exregional_fire_emission.sh +++ b/scripts/exregional_fire_emission.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_get_extrn_mdl_files.sh b/scripts/exregional_get_extrn_mdl_files.sh index ffdcceef00..018a30c285 100755 --- a/scripts/exregional_get_extrn_mdl_files.sh +++ b/scripts/exregional_get_extrn_mdl_files.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_make_grid.sh b/scripts/exregional_make_grid.sh index 88472a8343..276b5efec4 100755 --- a/scripts/exregional_make_grid.sh +++ b/scripts/exregional_make_grid.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_make_ics.sh b/scripts/exregional_make_ics.sh index d52bf25ec4..b42c086624 100755 --- a/scripts/exregional_make_ics.sh +++ b/scripts/exregional_make_ics.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_make_lbcs.sh b/scripts/exregional_make_lbcs.sh index 08cd0dda18..72f9369ff6 100755 --- a/scripts/exregional_make_lbcs.sh +++ b/scripts/exregional_make_lbcs.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_make_orog.sh b/scripts/exregional_make_orog.sh index f595d405a0..4fcf34a4ad 100755 --- a/scripts/exregional_make_orog.sh +++ b/scripts/exregional_make_orog.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_make_sfc_climo.sh b/scripts/exregional_make_sfc_climo.sh index d641fd6267..4e47aca9d1 100755 --- a/scripts/exregional_make_sfc_climo.sh +++ b/scripts/exregional_make_sfc_climo.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- @@ -58,11 +58,11 @@ export OMP_STACKSIZE=${OMP_STACKSIZE_MAKE_SFC_CLIMO} # #----------------------------------------------------------------------- # -# Are these machine dependent?? +# machine dependent ulimit commands # #----------------------------------------------------------------------- # -ulimit -s unlimited +eval ${PRE_TASK_CMDS} # #----------------------------------------------------------------------- # diff --git a/scripts/exregional_nexus_emission.sh b/scripts/exregional_nexus_emission.sh index 8c0f904b94..d1153d95b7 100755 --- a/scripts/exregional_nexus_emission.sh +++ b/scripts/exregional_nexus_emission.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_nexus_gfs_sfc.sh b/scripts/exregional_nexus_gfs_sfc.sh index 01207e5666..c34d2c30ae 100755 --- a/scripts/exregional_nexus_gfs_sfc.sh +++ b/scripts/exregional_nexus_gfs_sfc.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_nexus_post_split.sh b/scripts/exregional_nexus_post_split.sh index a2e2a63b2a..390e0dcce6 100755 --- a/scripts/exregional_nexus_post_split.sh +++ b/scripts/exregional_nexus_post_split.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_point_source.sh b/scripts/exregional_point_source.sh index 8eee83337c..aeec8f3925 100755 --- a/scripts/exregional_point_source.sh +++ b/scripts/exregional_point_source.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_post_stat_o3.sh b/scripts/exregional_post_stat_o3.sh index 266cfe67f9..94306d7336 100755 --- a/scripts/exregional_post_stat_o3.sh +++ b/scripts/exregional_post_stat_o3.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_post_stat_pm25.sh b/scripts/exregional_post_stat_pm25.sh index 47ee326fb4..dc054b87a3 100755 --- a/scripts/exregional_post_stat_pm25.sh +++ b/scripts/exregional_post_stat_pm25.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_pre_post_stat.sh b/scripts/exregional_pre_post_stat.sh index 33bbd75a41..44f4637684 100755 --- a/scripts/exregional_pre_post_stat.sh +++ b/scripts/exregional_pre_post_stat.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_fcst.sh b/scripts/exregional_run_fcst.sh index 6dfad9673b..c5519d923c 100755 --- a/scripts/exregional_run_fcst.sh +++ b/scripts/exregional_run_fcst.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_met_genensprod_or_ensemblestat.sh b/scripts/exregional_run_met_genensprod_or_ensemblestat.sh index 9ccb2d1b25..1041c9b994 100755 --- a/scripts/exregional_run_met_genensprod_or_ensemblestat.sh +++ b/scripts/exregional_run_met_genensprod_or_ensemblestat.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_met_gridstat_or_pointstat_vx.sh b/scripts/exregional_run_met_gridstat_or_pointstat_vx.sh index 46c93ee299..f71930dfb0 100755 --- a/scripts/exregional_run_met_gridstat_or_pointstat_vx.sh +++ b/scripts/exregional_run_met_gridstat_or_pointstat_vx.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensmean.sh b/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensmean.sh index bf3a8280a1..d50827a52c 100755 --- a/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensmean.sh +++ b/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensmean.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensprob.sh b/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensprob.sh index 89c128eeed..89b8ae98b3 100755 --- a/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensprob.sh +++ b/scripts/exregional_run_met_gridstat_or_pointstat_vx_ensprob.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_met_pb2nc_obs.sh b/scripts/exregional_run_met_pb2nc_obs.sh index be0f0a2b91..07e08f31db 100755 --- a/scripts/exregional_run_met_pb2nc_obs.sh +++ b/scripts/exregional_run_met_pb2nc_obs.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_met_pcpcombine.sh b/scripts/exregional_run_met_pcpcombine.sh index cb047163cb..b45a54330e 100755 --- a/scripts/exregional_run_met_pcpcombine.sh +++ b/scripts/exregional_run_met_pcpcombine.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_post.sh b/scripts/exregional_run_post.sh index 76a1196329..edf35ef2a4 100755 --- a/scripts/exregional_run_post.sh +++ b/scripts/exregional_run_post.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- diff --git a/scripts/exregional_run_prdgen.sh b/scripts/exregional_run_prdgen.sh index bf7a9b8e83..8fc72dff1c 100755 --- a/scripts/exregional_run_prdgen.sh +++ b/scripts/exregional_run_prdgen.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #----------------------------------------------------------------------- @@ -212,7 +212,7 @@ sed -n -e '1,500p' $DATAprdgen/prslevf${fhr}.txt >& $DATAprdgen/hi_pr_1.txt sed -n -e '501,$p' $DATAprdgen/prslevf${fhr}.txt >& $DATAprdgen/hi_pr_2.txt # Create script to execute production generation tasks in parallel using CFP -echo "#!/bin/bash" > $DATAprdgen/poescript_${fhr} +echo "#!/usr/bin/env bash" > $DATAprdgen/poescript_${fhr} echo "export DATA=${DATAprdgen}" >> $DATAprdgen/poescript_${fhr} echo "export COMOUT=${COMOUT}" >> $DATAprdgen/poescript_${fhr} diff --git a/scripts/exregional_run_prdgen_subpiece.sh b/scripts/exregional_run_prdgen_subpiece.sh index 0b0b08630a..37e52c46fe 100755 --- a/scripts/exregional_run_prdgen_subpiece.sh +++ b/scripts/exregional_run_prdgen_subpiece.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash ################################################################################ #### UNIX Script Documentation Block # . . diff --git a/sorc/patch_macos_arm64_sorc_cmakelists.txt b/sorc/patch_macos_arm64_sorc_cmakelists.txt new file mode 100644 index 0000000000..582a3cb068 --- /dev/null +++ b/sorc/patch_macos_arm64_sorc_cmakelists.txt @@ -0,0 +1,18 @@ +--- sorc/CMakeLists.txt 2023-10-26 16:06:03 ++++ sorc/CMakeLists.txt 2023-10-28 10:01:58 +@@ -45,9 +45,12 @@ + list(APPEND UFS_WEATHER_MODEL_ARGS + "-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}" + "-DCCPP_SUITES=${CCPP_SUITES}" +- "-DCMAKE_C_COMPILER=${MPI_C_COMPILER}" +- "-DCMAKE_CXX_COMPILER=${MPI_CXX_COMPILER}" +- "-DCMAKE_Fortran_COMPILER=${MPI_Fortran_COMPILER}" ++ "-DCMAKE_C_COMPILER=${C_COMPILER}" ++ "-DCMAKE_CXX_COMPILER=${CXX_COMPILER}" ++ "-DCMAKE_Fortran_COMPILER=${Fortran_COMPILER}" ++ "-DCMAKE_MPI_C_COMPILER=${MPI_C_COMPILER}" ++ "-DCMAKE_MPI_CXX_COMPILER=${MPI_CXX_COMPILER}" ++ "-DCMAKE_MPI_Fortran_COMPILER=${MPI_Fortran_COMPILER}" + "-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}" + "-DCMAKE_MODULE_PATH=${MAPL_ROOT}/share/MAPL/cmake" + "-DNETCDF_DIR=$ENV{NETCDF}" diff --git a/ush/job_preamble.sh b/ush/job_preamble.sh index e243f31b37..30ee7e0867 100644 --- a/ush/job_preamble.sh +++ b/ush/job_preamble.sh @@ -91,7 +91,7 @@ export SENDWEB=$(boolify "${SENDWEB}") # #----------------------------------------------------------------------- # -if [ $subcyc -eq 0 ]; then +if [ ${subcyc:-0} -eq 0 ]; then export cycle="t${cyc}z" else export cycle="t${cyc}${subcyc}z" diff --git a/ush/link_fix.py b/ush/link_fix.py index fdd9a65f28..db2fa7a23c 100755 --- a/ush/link_fix.py +++ b/ush/link_fix.py @@ -12,7 +12,6 @@ print_info_msg, print_err_msg_exit, create_symlink_to_file, - define_macos_utilities, check_var_valid_value, flatten_dict, cd_vrfy, diff --git a/ush/machine/macos.yaml b/ush/machine/macos.yaml index f6d5e2a952..ac1993ded1 100644 --- a/ush/machine/macos.yaml +++ b/ush/machine/macos.yaml @@ -1,29 +1,36 @@ platform: - WORKFLOW_MANAGER: rocoto + WORKFLOW_MANAGER: none NCORES_PER_NODE: 8 TASKTHROTTLE: 1 - SCHED: slurm - TEST_CCPA_OBS_DIR: /Users/username/DATA/UFS/obs_data/ccpa/proc - TEST_MRMS_OBS_DIR: /Users/username/DATA/UFS/obs_data/mrms/proc - TEST_NDAS_OBS_DIR: /Users/username/DATA/UFS/obs_data/ndas/proc - DOMAIN_PREGEN_BASEDIR: /Users/username/DATA/UFS/FV3LAM_pregen + SCHED: none + TEST_CCPA_OBS_DIR: /Users/username/UFS_SRW_data/v2p2/obs_data/ccpa/proc + TEST_MRMS_OBS_DIR: /Users/username/UFS_SRW_data/v2p2/obs_data/mrms/proc + TEST_NDAS_OBS_DIR: /Users/username/UFS_SRW_data/v2p2/obs_data/ndas/proc + DOMAIN_PREGEN_BASEDIR: /Users/username/UFS_SRW_data/v2p2/FV3LAM_pregen RUN_CMD_FCST: mpirun -n ${PE_MEMBER01} RUN_CMD_POST: mpirun RUN_CMD_PRDGEN: mpirun RUN_CMD_SERIAL: time RUN_CMD_UTILS: mpirun - PRE_TASK_CMDS: '{ ulimit -a; ulimit -s unlimited; }' - TEST_EXTRN_MDL_SOURCE_BASEDIR: /Users/username/DATA/UFS/input_model_data - TEST_PREGEN_BASEDIR: /Users/username/DATA/UFS/FV3LAM_pregen - TEST_ALT_EXTRN_MDL_SYSBASEDIR_ICS: /Users/username/DATA/UFS/dummy_FV3GFS_sys_dir - TEST_ALT_EXTRN_MDL_SYSBASEDIR_LBCS: /Users/username/DATA/UFS/dummy_FV3GFS_sys_dir - FIXaer: /Users/username/DATA/UFS/fix/fix_aer - FIXgsm: /Users/username/DATA/UFS/fix/fix_am - FIXlut: /Users/username/DATA/UFS/fix/fix_lut - FIXorg: /Users/username/DATA/UFS/fix/fix_orog - FIXsfc: /Users/username/DATA/UFS/fix/fix_sfc_climo - FIXshp: /Users/username/DATA/UFS/NaturalEarth + PRE_TASK_CMDS: '{ ulimit -a; ulimit -S -s unlimited; }' + TEST_EXTRN_MDL_SOURCE_BASEDIR: /Users/username/UFS_SRW_data/v2p2/input_model_data + TEST_PREGEN_BASEDIR: /Users/username/UFS_SRW_data/v2p2/FV3LAM_pregen + TEST_ALT_EXTRN_MDL_SYSBASEDIR_ICS: /Users/username/UFS_SRW_data/v2p2/dummy_FV3GFS_sys_dir + TEST_ALT_EXTRN_MDL_SYSBASEDIR_LBCS: /Users/username/UFS_SRW_data/v2p2/dummy_FV3GFS_sys_dir + FIXaer: /Users/username/UFS_SRW_data/v2p2/fix/fix_aer + FIXgsm: /Users/username/UFS_SRW_data/v2p2/fix/fix_am + FIXlut: /Users/username/UFS_SRW_data/v2p2/fix/fix_lut + FIXorg: /Users/username/UFS_SRW_data/v2p2/fix/fix_orog + FIXsfc: /Users/username/UFS_SRW_data/v2p2/fix/fix_sfc_climo + FIXshp: /Users/username/UFS_SRW_data/v2p2/NaturalEarth EXTRN_MDL_DATA_STORES: aws nomads data: ics_lbcs: - FV3GFS: /Users/username/DATA/UFS/FV3GFS + FV3GFS: /Users/username/UFS_SRW_data/v2p2/input_model_data/FV3GFS + nemsio: /Users/username/UFS_SRW_data/v2p2/input_model_data/FV3GFS/nemsio/${yyyymmdd}${hh} + grib2: /Users/username/UFS_SRW_data/v2p2/input_model_data/FV3GFS/grib2/${yyyymmdd}${hh} + netcdf: /Users/username/UFS_SRW_data/v2p2/input_model_data/FV3GFS/netcdf/${yyyymmdd}${hh} + NAM: /Users/username/UFS_SRW_data/v2p2/input_model_data/NAM/${yyyymmdd}${hh} + HRRR: /Users/username/UFS_SRW_data/v2p2/input_model_data/HRRR/${yyyymmdd}${hh} + RAP: /Users/username/UFS_SRW_data/v2p2/input_model_data/RAP/${yyyymmdd}${hh} + GSMGFS: /Users/username/UFS_SRW_data/v2p2/input_model_data/GSMGFS/${yyyymmdd}${hh} diff --git a/ush/preamble.sh b/ush/preamble.sh index 0572905d55..85643541ec 100644 --- a/ush/preamble.sh +++ b/ush/preamble.sh @@ -56,9 +56,9 @@ if [[ -z ${FUNCNAME[0]} ]]; then rc=${3} # Calculate the elapsed time - end_time=$(date +%s) + end_time=$(${DATE_UTIL} +%s) elapsed_sec=$((end_time - start_time)) - elapsed=$(date -d@${elapsed_sec} -u +%H:%M:%S) + elapsed=$(${DATE_UTIL} -d@${elapsed_sec} -u +%H:%M:%S) # Announce the script has ended, then pass the error code up echo "End ${script} at $(date -u) with error code ${rc:-0} (time elapsed: ${elapsed})" @@ -66,13 +66,13 @@ if [[ -z ${FUNCNAME[0]} ]]; then } # Record the start time so we can calculate the elapsed time later - start_time=$(date +%s) + start_time=$(${DATE_UTIL} +%s) # Get the base name of the calling script _calling_script=$(basename ${BASH_SOURCE[1]}) # Announce the script has begun - echo "Begin ${_calling_script} at $(date -u)" + echo "Begin ${_calling_script} at $(${DATE_UTIL} -u)" # Place the postamble in a trap so it is always called no matter how the script exits trap "postamble ${_calling_script} ${start_time} \$?" EXIT diff --git a/ush/python_utils/filesys_cmds_vrfy.py b/ush/python_utils/filesys_cmds_vrfy.py index ca986f9693..629d8eb7e4 100644 --- a/ush/python_utils/filesys_cmds_vrfy.py +++ b/ush/python_utils/filesys_cmds_vrfy.py @@ -2,7 +2,7 @@ import os from .print_msg import print_err_msg_exit - +from .define_macos_utilities import define_macos_utilities def cmd_vrfy(cmd, *args): """Execute system command @@ -38,7 +38,9 @@ def rm_vrfy(*args): def ln_vrfy(*args): - return cmd_vrfy("ln", *args) + define_macos_utilities() + cmd_ln = os.getenv("LN_UTIL") + return cmd_vrfy(cmd_ln, *args) def mkdir_vrfy(*args): diff --git a/ush/wrappers/qsub_job.sh b/ush/wrappers/qsub_job.sh index 8cd49aa57c..c4b6895981 100755 --- a/ush/wrappers/qsub_job.sh +++ b/ush/wrappers/qsub_job.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash #PBS -A XXXXXXXXX #PBS -q regular #PBS -l select=1:mpiprocs=24:ncpus=24 diff --git a/ush/wrappers/run_fcst.sh b/ush/wrappers/run_fcst.sh index 93cc009b58..7450de7cc5 100755 --- a/ush/wrappers/run_fcst.sh +++ b/ush/wrappers/run_fcst.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_get_ics.sh b/ush/wrappers/run_get_ics.sh index 4cba2a86e5..0ee521a67d 100755 --- a/ush/wrappers/run_get_ics.sh +++ b/ush/wrappers/run_get_ics.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_get_lbcs.sh b/ush/wrappers/run_get_lbcs.sh index 3447d9f224..543ab6e47d 100755 --- a/ush/wrappers/run_get_lbcs.sh +++ b/ush/wrappers/run_get_lbcs.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_gridensvx.sh b/ush/wrappers/run_gridensvx.sh index 6134e6f33a..d3cb0c12ae 100755 --- a/ush/wrappers/run_gridensvx.sh +++ b/ush/wrappers/run_gridensvx.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash # Stand-alone script to run grid-to-grid ensemble verification export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" diff --git a/ush/wrappers/run_gridvx.sh b/ush/wrappers/run_gridvx.sh index 9b6f0e3979..9a5d55b046 100755 --- a/ush/wrappers/run_gridvx.sh +++ b/ush/wrappers/run_gridvx.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash # Stand-alone script to run grid-to-grid verification export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" diff --git a/ush/wrappers/run_make_grid.sh b/ush/wrappers/run_make_grid.sh index deff13c0ee..2d55beaf94 100755 --- a/ush/wrappers/run_make_grid.sh +++ b/ush/wrappers/run_make_grid.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_make_ics.sh b/ush/wrappers/run_make_ics.sh index b3e4333d9d..5c629722fc 100755 --- a/ush/wrappers/run_make_ics.sh +++ b/ush/wrappers/run_make_ics.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_make_lbcs.sh b/ush/wrappers/run_make_lbcs.sh index 2cd59c47a2..27c94c127f 100755 --- a/ush/wrappers/run_make_lbcs.sh +++ b/ush/wrappers/run_make_lbcs.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_make_orog.sh b/ush/wrappers/run_make_orog.sh index 81bdd5787b..5f02ff9599 100755 --- a/ush/wrappers/run_make_orog.sh +++ b/ush/wrappers/run_make_orog.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_make_sfc_climo.sh b/ush/wrappers/run_make_sfc_climo.sh index 455a8fdf45..fab33f75d6 100755 --- a/ush/wrappers/run_make_sfc_climo.sh +++ b/ush/wrappers/run_make_sfc_climo.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_pointensvx.sh b/ush/wrappers/run_pointensvx.sh index 87ad3e2e30..ca1c63952f 100755 --- a/ush/wrappers/run_pointensvx.sh +++ b/ush/wrappers/run_pointensvx.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash # Stand-alone script to run grid-to-point ensemble verification export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" diff --git a/ush/wrappers/run_pointvx.sh b/ush/wrappers/run_pointvx.sh index d59a57ed5f..e6754d8e78 100755 --- a/ush/wrappers/run_pointvx.sh +++ b/ush/wrappers/run_pointvx.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash # Stand-alone script to run grid-to-point verification export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" diff --git a/ush/wrappers/run_post.sh b/ush/wrappers/run_post.sh index 4a4a57513e..46ef104365 100755 --- a/ush/wrappers/run_post.sh +++ b/ush/wrappers/run_post.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_process_lightning.sh b/ush/wrappers/run_process_lightning.sh index d183d3adcd..1b6668136e 100755 --- a/ush/wrappers/run_process_lightning.sh +++ b/ush/wrappers/run_process_lightning.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/run_process_radarref.sh b/ush/wrappers/run_process_radarref.sh index 694b79b4f8..39475b81be 100755 --- a/ush/wrappers/run_process_radarref.sh +++ b/ush/wrappers/run_process_radarref.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash export GLOBAL_VAR_DEFNS_FP="${EXPTDIR}/var_defns.sh" set -xa source ${GLOBAL_VAR_DEFNS_FP} diff --git a/ush/wrappers/sq_job.sh b/ush/wrappers/sq_job.sh index a9fa3c1805..6a817b3196 100755 --- a/ush/wrappers/sq_job.sh +++ b/ush/wrappers/sq_job.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash #SBATCH -e /path/to/exptdir/log/run_make_grid.log # NEED TO SET #SBATCH --account=XXXXXXXXX #SBATCH --qos=batch