From 6d6c6af4a247a52454b34e9051c79f4710fc76eb Mon Sep 17 00:00:00 2001 From: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> Date: Fri, 1 Mar 2024 16:45:39 -0500 Subject: [PATCH] [develop]: Add Contributor's Guide to documentation (#1046) This PR adds a Contributor's Guide to the docs alongside the User's Guide. * The Contributor's Guide includes general information on use of Git submodules in the UFS. This information can be adapted in a future PR to be more SRW-specific based on user needs/requests and any training we provide. * This PR also configures the docs so that Technical Documentation can be easily added at a later date. --------- Co-authored-by: RatkoVasic-NOAA <37597874+RatkoVasic-NOAA@users.noreply.github.com> --- .readthedocs.yaml | 4 +- .../code-configuration-standards.rst | 61 ++++ doc/ContribGuide/contributing.rst | 340 ++++++++++++++++++ doc/ContribGuide/documentation.rst | 72 ++++ doc/ContribGuide/git-submodules.rst | 225 ++++++++++++ doc/ContribGuide/index.rst | 13 + doc/ContribGuide/introduction.rst | 31 ++ doc/ContribGuide/testing.rst | 70 ++++ doc/INSTALL | 2 +- doc/{UsersGuide => }/Makefile | 7 +- doc/{UsersGuide => }/README | 0 .../BackgroundInfo/CCPPUpdates.rst | 0 .../BackgroundInfo/Components.rst | 2 +- .../BackgroundInfo/Introduction.rst | 45 ++- .../BackgroundInfo/TechnicalOverview.rst | 4 +- .../{source => }/BackgroundInfo/index.rst | 0 .../BuildingRunningTesting/AQM.rst | 0 .../BuildingRunningTesting/BuildSRW.rst | 0 .../ContainerQuickstart.rst | 0 .../DefaultVarsTable.rst | 0 .../BuildingRunningTesting/Quickstart.rst | 0 .../BuildingRunningTesting/RunSRW.rst | 0 .../BuildingRunningTesting/Tutorial.rst | 0 .../BuildingRunningTesting/VXCases.rst | 0 .../BuildingRunningTesting/WE2Etests.rst | 4 +- .../BuildingRunningTesting/index.rst | 0 .../CustomizingTheWorkflow/ConfigWorkflow.rst | 6 +- .../CustomizingTheWorkflow/DefineWorkflow.rst | 0 .../InputOutputFiles.rst | 9 +- .../CustomizingTheWorkflow/LAMGrids.rst | 0 .../CustomizingTheWorkflow/TemplateVars.rst | 0 .../CustomizingTheWorkflow/index.rst | 0 doc/UsersGuide/{source => }/Reference/FAQ.rst | 0 .../{source => }/Reference/Glossary.rst | 4 +- .../{source => }/Reference/RocotoInfo.rst | 0 .../{source => }/Reference/index.rst | 0 doc/UsersGuide/{source => }/SSHIntro.rst | 0 doc/UsersGuide/index.rst | 10 + .../source/_static/theme_overrides.css | 24 -- doc/UsersGuide/source/index.rst | 16 - .../source => }/_static/custom.css | 0 doc/_static/theme_overrides.css | 26 ++ .../source => }/_templates/.gitignore | 0 doc/{UsersGuide/source => }/conf.py | 90 ++--- doc/index.rst | 9 + doc/{UsersGuide => }/make.bat | 0 doc/{UsersGuide/source => }/references.bib | 0 doc/{UsersGuide => }/requirements.in | 0 doc/{UsersGuide => }/requirements.txt | 4 +- .../source => }/tables/SRW_NATLEV_table.csv | 0 .../source => }/tables/SRW_NATLEV_table.rst | 0 .../source => }/tables/SRW_PRSLEV_table.csv | 0 .../source => }/tables/SRW_PRSLEV_table.rst | 0 doc/{UsersGuide/source => }/tables/Tests.csv | 0 doc/{UsersGuide/source => }/tables/Tests.rst | 0 doc/tables/code-managers.csv | 21 ++ .../source => }/tables/fix_file_list.rst | 0 57 files changed, 974 insertions(+), 125 deletions(-) create mode 100644 doc/ContribGuide/code-configuration-standards.rst create mode 100644 doc/ContribGuide/contributing.rst create mode 100644 doc/ContribGuide/documentation.rst create mode 100644 doc/ContribGuide/git-submodules.rst create mode 100644 doc/ContribGuide/index.rst create mode 100644 doc/ContribGuide/introduction.rst create mode 100644 doc/ContribGuide/testing.rst rename doc/{UsersGuide => }/Makefile (84%) rename doc/{UsersGuide => }/README (100%) rename doc/UsersGuide/{source => }/BackgroundInfo/CCPPUpdates.rst (100%) rename doc/UsersGuide/{source => }/BackgroundInfo/Components.rst (99%) rename doc/UsersGuide/{source => }/BackgroundInfo/Introduction.rst (78%) rename doc/UsersGuide/{source => }/BackgroundInfo/TechnicalOverview.rst (94%) rename doc/UsersGuide/{source => }/BackgroundInfo/index.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/AQM.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/BuildSRW.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/ContainerQuickstart.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/DefaultVarsTable.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/Quickstart.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/RunSRW.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/Tutorial.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/VXCases.rst (100%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/WE2Etests.rst (99%) rename doc/UsersGuide/{source => }/BuildingRunningTesting/index.rst (100%) rename doc/UsersGuide/{source => }/CustomizingTheWorkflow/ConfigWorkflow.rst (99%) rename doc/UsersGuide/{source => }/CustomizingTheWorkflow/DefineWorkflow.rst (100%) rename doc/UsersGuide/{source => }/CustomizingTheWorkflow/InputOutputFiles.rst (98%) rename doc/UsersGuide/{source => }/CustomizingTheWorkflow/LAMGrids.rst (100%) rename doc/UsersGuide/{source => }/CustomizingTheWorkflow/TemplateVars.rst (100%) rename doc/UsersGuide/{source => }/CustomizingTheWorkflow/index.rst (100%) rename doc/UsersGuide/{source => }/Reference/FAQ.rst (100%) rename doc/UsersGuide/{source => }/Reference/Glossary.rst (98%) rename doc/UsersGuide/{source => }/Reference/RocotoInfo.rst (100%) rename doc/UsersGuide/{source => }/Reference/index.rst (100%) rename doc/UsersGuide/{source => }/SSHIntro.rst (100%) create mode 100644 doc/UsersGuide/index.rst delete mode 100644 doc/UsersGuide/source/_static/theme_overrides.css delete mode 100644 doc/UsersGuide/source/index.rst rename doc/{UsersGuide/source => }/_static/custom.css (100%) create mode 100644 doc/_static/theme_overrides.css rename doc/{UsersGuide/source => }/_templates/.gitignore (100%) rename doc/{UsersGuide/source => }/conf.py (85%) create mode 100644 doc/index.rst rename doc/{UsersGuide => }/make.bat (100%) rename doc/{UsersGuide/source => }/references.bib (100%) rename doc/{UsersGuide => }/requirements.in (100%) rename doc/{UsersGuide => }/requirements.txt (97%) rename doc/{UsersGuide/source => }/tables/SRW_NATLEV_table.csv (100%) rename doc/{UsersGuide/source => }/tables/SRW_NATLEV_table.rst (100%) rename doc/{UsersGuide/source => }/tables/SRW_PRSLEV_table.csv (100%) rename doc/{UsersGuide/source => }/tables/SRW_PRSLEV_table.rst (100%) rename doc/{UsersGuide/source => }/tables/Tests.csv (100%) rename doc/{UsersGuide/source => }/tables/Tests.rst (100%) create mode 100644 doc/tables/code-managers.csv rename doc/{UsersGuide/source => }/tables/fix_file_list.rst (100%) diff --git a/.readthedocs.yaml b/.readthedocs.yaml index b6afe96c93..c8ce6064b2 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -17,7 +17,7 @@ build: # Build documentation in the docs/ directory with Sphinx sphinx: - configuration: doc/UsersGuide/source/conf.py + configuration: doc/conf.py # If using Sphinx, optionally build your docs in additional formats such as PDF # formats: @@ -26,7 +26,7 @@ sphinx: # Optionally declare the Python requirements required to build your docs python: install: - - requirements: doc/UsersGuide/requirements.txt + - requirements: doc/requirements.txt submodules: include: diff --git a/doc/ContribGuide/code-configuration-standards.rst b/doc/ContribGuide/code-configuration-standards.rst new file mode 100644 index 0000000000..4cc043a9c2 --- /dev/null +++ b/doc/ContribGuide/code-configuration-standards.rst @@ -0,0 +1,61 @@ +================================== +Code and Configuration Standards +================================== + +General Policies +================== + +* Platform-specific settings should be handled only through configuration and modulefiles, not in code or scripts. +* For changes to the ``scripts``, ``ush``, or ``jobs`` directories, developers should follow the :nco:`NCO Guidelines <>` for what is incorporated into each layer. +* Developers should ensure that their contributions work with the most recent version of the ``ufs-srweather-app``, including all the specific up-to-date hashes of each subcomponent. +* Modifications should not break any existing supported capabilities on any supported platforms. +* Update the RST documentation files where appropriate as part of the PR. If necessary, contributors may update the documentation in a subsequent PR. In these cases, the contributor should :srw-repo:`open an issue ` reflecting the need for documentation and include the issue number and explanation in the Documentation section of their initial PR. +* Binary files will no longer be merged into the ``develop`` branch. A binary file is defined as a "non-text" file and can include ``*.png``, ``*.gif``, ``*.jp*g``, ``*.tiff``, ``*.tar``, ``*.tgz``, ``*.gz``, ``*.mod``, ``*.o``, and executables. If a binary file needs to be staged in the ``ufs-srweather-app`` repository, please add it to the wiki's repository. The command to clone the ``ufs-srweather-app``'s wiki repository is ``git clone https://github.com/ufs-community/ufs-srweather-app.wiki.git``. Users with write access to the wiki repository can add the files here and link them to the documentation as needed. Users who do not have write access to the wiki repository should reach out to @MichaelLueken and/or note this in their pull request so that the files can be added. + +SRW Application Guidelines +============================ + + +**General Coding Standards:** + +* The ``ufs-srweather-app`` repository must not contain source code for compiled programs. Only scripts and configuration files should reside in this repository. +* All bash scripts must explicitly be ``#!/bin/bash`` scripts. They should *not* be login-enabled (i.e., scripts should *not* use the ``-l`` flag). +* MacOS does not have all Linux utilities by default. Developers should ensure that they do not break any MacOS capabilities with their contribution. +* All code must be indented appropriately and conform to the style of existing scripts (e.g., local variables should be lowercase, global variables should be uppercase). + +**External Components** + +* All externals live in a single ``Externals.cfg`` file. +* Only a single hash will be maintained for any given external code base. All externals should point to this static hash (not to the top of a branch). +* All new entries in ``Externals.cfg`` must point only to authoritative repositories. In other words, entries must point to either a `ufs-community GitHub organization `__ repository or another NOAA project organization repository. + + * Temporary exceptions are made for a PR into the ``develop`` branch of ``ufs-srweather-app`` that is dependent on another PR. When the component PR is merged, the contributor must update the corresponding ``ufs-srweather-app`` PR with the hash of the component's authoritative repository. + +**Build System** + +* Each component must build with CMake +* Each component must build with Intel compilers on official :srw-wiki:`Level 1 ` platforms and with GNU or Intel compilers on other platforms. +* Each component must have a mechanism for platform independence (i.e., no hard-coded machine-specific settings outside of established environment, configuration, and modulefiles). +* Each component must build with the standard supported NCEPLIBS environment (currently `spack-stack `__). + +**Modulefiles** + +* All official platforms should have a modulefile that can be sourced to provide the appropriate Python packages and other settings for the platform. +* Each SRW component must build using the common modules located in the ``modulefiles/srw_common`` file. + + +Workflow Coding Standards +-------------------------- + +**Python Coding Standards:** + + * All new Python workflow contributions should come with an appropriate environment YAML file (similar to ``environment.yaml``, ``graphics_environment.yaml``, and ``aqm_environment.yaml``). + * Keep the use of external Python packages to a minimum for necessary workflow tasks. Currently, these include ``f90nml``, ``pyyaml``, and ``Jinja2``. + +**Workflow Design:** Follow the :nco:`NCO Guidelines <>` for what is incorporated in each layer of the workflow. This is particularly important in the ``scripts`` directory. + +**Management of the Configuration File:** New configurable options must be consistent with existing configurable options and be documented in :srw-repo:`UsersGuide/CustomizingTheWorkflow/ConfigWorkflow.rst `. Add necessary checks on acceptable options where applicable. Add appropriate default values in ``config_defaults.yaml``. + +**Management of Template Files:** If a new configurable option is required in an existing template, it must be handled similarly to its counterparts in the scripts that fill in the template. For example, if a new type of namelist is introduced for a new application component, it should make use of the existing ``jinja`` framework for populating namelist settings. + +**Namelist Management:** Namelists in ``ufs-srweather-app`` are generated using a Python tool and managed by setting YAML configuration parameters. This allows for the management of multiple configuration settings with maximum flexibility and minimum duplication of information. \ No newline at end of file diff --git a/doc/ContribGuide/contributing.rst b/doc/ContribGuide/contributing.rst new file mode 100644 index 0000000000..ed1671363e --- /dev/null +++ b/doc/ContribGuide/contributing.rst @@ -0,0 +1,340 @@ +============================ +Contributing to the SRW App +============================ + +Fork and PR Overview +===================== + +.. note:: + + Thank you to the Unified Workflow (UW) team for allowing us to adapt their Fork and PR Model overview for use in the SRW App. The original can be viewed in the `uwtools` :uw:`documentation `. + +Contributions to the ``ufs-srweather-app`` project are made via a :github-docs:`Fork` and :github-docs:`Pull Request (PR)` model. GitHub provides a thorough description of this contribution model in their `Contributing to a project` :github-docs:`Quickstart`, but the steps, with respect to ``ufs-srweather-app`` contributions, can be summarized as: + +#. :github-docs:`Create an issue ` to document proposed changes. +#. :github-docs:`Fork` the :srw-repo:`ufs-srweather-app repository<>` into your personal GitHub account. +#. :github-docs:`Clone` your fork onto your development system. +#. :github-docs:`Create a branch` in your clone for your changes. All development should take place on a branch, *not* on ``develop``. +#. :github-docs:`Make, commit, and push changes` in your clone / to your fork. +#. When your work is complete, :github-docs:`create a pull request (PR)` to merge your changes. + +For future contributions, you may delete and then recreate your fork or configure the official ``ufs-srweather-app`` repository as a :github-docs:`remote repository` on your clone and :github-docs:`sync upstream changes` to stay up-to-date with the official repository. + + +Development and Testing Process +================================= + +#. **Create issue:** Open an :srw-repo:`issue ` in the ``ufs-srweather-app`` to document proposed changes. See :ref:`Opening an Issue ` for detailed instructions. +#. **Fork & Clone the SRW App:** :github-docs:`Fork` the :srw-repo:`ufs-srweather-app repository<>` into your personal GitHub account and :github-docs:`clone` your fork onto your development system if you have not already done so. +#. **Create a branch:** in your clone for your changes. All development should take place on a branch, not on ``develop``. Branches should be named as follows, where ``[name]`` is a one-word description of the branch: + + * ``bugfix/[name]``: Fixes a demonstrably incorrect portion of code + * ``feature/[name]``: Adds a new feature to the code or improves an existing portion of the code + * ``text/[name]``: Changes elements of the repository that do not impact the compiled code in any way (e.g., changes to README, documentation, comments, changing quoted Registry elements, white space alignment). + +#. **Development:** Perform and test changes in the feature branch (not on ``develop``!). Document work in the issue and mention the issue number in commit messages to link your work to the issue (e.g., ``commit -m "Issue #23 - "``). Document changes to the workflow and capabilities in the RST files so that the SRW App documentation stays up-to-date. +#. **Testing:** Test code modifications on as many platforms as possible, and request help with further testing from the code management team when unable to test on all Level 1 platforms. The bare minimum testing required before opening a PR is to run the fundamental (:srw-repo:`tests/WE2E/machine_suites/fundamental `) tests on at least one supported machine (additional testing from the comprehensive suite might be required, depending on the nature of the change). To run the fundamental tests manually, please use the following command in the ``tests/WE2E`` directory: + + .. code-block:: console + + ./run_WE2E_tests.py -t=fundamental -m=your_machine -a=your_account + + where ``your_machine`` is the Tier-1 machine you are running the tests on, and ``your_account`` is the account you charge your computational resources to. See section :numref:`Section %s ` for more detail on SRW App testing. + +#. **Pull Request:** When your work is complete, :github-docs:`create a pull request` to merge your changes. When a PR is initiated, the :ref:`PR template ` autofills. Developers should use the template to provide information about the PR in the proper fields. See the guidelines in the :ref:`Making a Pull Request ` section for more details on making a good pull request. +#. **Merge** - When review and testing are complete, a code manager will merge the PR into ``develop``. PRs that are not ready for merging should have a "Work in Progress" label on them. Users who lack the permissions required to add the label can request in their PR that a code manager do so. +#. **Cleanup** - After the PR is merged, the code developer should delete the branch on their fork and close the issue. Feature branches are intended to be short-lived, concentrated on code with one sole purpose, and applicable to a single PR. A new feature branch should be created when subsequent code development continues. + +.. note:: + + Communication with code managers and the :ref:`repository code management team ` throughout the process is encouraged. + +.. _open-issue: + +Opening an Issue +================= + +All changes to ``ufs-srweather-app`` should be associated with a :srw-repo:`GitHub Issue `. Developers should search the existing issues in the ``ufs-srweather-app`` repository before beginning their work. If an issue does not exist for the work they are doing, they should create one prior to opening a new pull request. If an issue does exist, developers should be sure to collaborate to avoid duplicative work. + +To open an issue, click on :srw-repo:`"New Issue"` within the ``ufs-srweather-app`` GitHub repository. + +Choose from four options: + +#. :srw-repo:`Bug Report `: Report specific problems ("bugs") in the code using the following template: + + .. code-block:: console + + + + Your bug may already be reported! + Please search on the [Issue tracker](https://github.com/ufs-community/ufs-srweather-app/issues) before creating a new issue. + If an issue already exists, please use that issue to add any additional information. + + ## Expected behavior + + + ## Current behavior + + + ## Machines affected + + + + ## Steps To Reproduce + + + ## Detailed Description of Fix (optional) + + + ## Additional Information (optional) + + + ## Possible Implementation (optional) + + + ## Output (optional) + + +#. :srw-repo:`Feature Request `: New features and feature enhancements fall under this category. Propose features and enhancements using the following template. Optional sections may be deleted. + + .. code-block:: console + + + + Your issue may already be reported! + Please search on the [Issue tracker](https://github.com/ufs-community/ufs-srweather-app/issues) before creating a new issue. If an issue already exists, please use that issue to add any additional information. + + ## Description + + + + + ## Solution + + + ## Requirements** + + + ## Acceptance Criteria (Definition of Done) + + + ## Dependencies (optional) + + + + ## Alternative Solutions (optional) + + + +#. :srw-repo:`Text-Only Changes `: Propose text-only changes using the "Text-only request" template. Optional sections may be deleted. + + .. code-block:: console + + ## Description + + + ## Solution + + + ## Alternatives (optional) + + + ## Related to (optional) + + +#. :srw-repo:`Other `: Open a blank issue, and use the "Feature Request" template above as a starting point to describe the issue. + +For all issue reports, indicate whether this is: + #. A problem that you plan to work on and submit a PR for + #. A problem that you will **not** work on but that requires attention + #. A suggested improvement + +Additionally, please add a priority label to the issue (low, medium, or high priority). If you are unable to add labels to your issues, please request that a code manager add a priority label for you. + + * **High priority:** Issues related to a bug fix, a failing test configuration, or an update required for a release (either an operational implementation or public release). + * **Medium priority:** New features that are not required immediately for either an implementation or release + * **Low priority:** Refactoring work or other work that does not seem to be medium or high priority. + +If you are unable to work on the issue and require assistance through :term:`EPIC`, please make sure to include the ``EPIC Support Requested`` label. If the ``EPIC Support Requested`` label is added to a ``high priority`` issue, it might take some time before EPIC will work on the issue, since EPIC management needs to account for and prioritize these issues. However, after seeing that EPIC is required for high priority issues, management will adapt and allocate the necessary resources to assist. After filling out the issue report, click on "Submit new issue." + + +.. _make-pr: + +Making a Pull Request +====================== + +All changes to the SRW App ``develop`` branch should be handled via GitHub’s "Pull Request" (PR) functionality from a branch in the developer's fork. When creating your PR, please follow these guidelines, specific to the ``ufs-srweather-app`` project: + +* Ensure that your PR is targeting the base repository ``ufs-community/ufs-srweather-app`` and an appropriate base branch (usually ``develop``). +* **Complete PR template.** Your PR will appear pre-populated with a :ref:`template ` that you should complete. Provide an informative synopsis of your contribution, then mark appropriate checklist items by placing an "X" between their square brackets. You may tidy up the description by removing boilerplate text and non-selected checklist items. View :ref:`useful PR template guidance ` and information on :ref:`best practices ` for completing each section below. +* **Create draft PR.** Use the pull-down arrow on the green button below the description to initially create a :github-docs:`draft pull request`. + + * Once your draft PR is open, visit its *Files changed* tab and add comments to any lines of code where you think reviewers will benefit from more explanation. Try to save time by proactively answering questions you suspect reviewers will ask. + +* **Open PR.** Once your draft PR is marked up with your comments and ready for review, return to the *Conversation* tab and click the *Ready for review* button. + + * A default set of reviewers will automatically be added to your PR. You may add or request others, if appropriate. Pull requests will be reviewed and approved by at least two code reviewers, at least one of whom must be a code manager. Reviewers may make comments, ask questions, or request changes on your PR. Respond to these as needed, making commits in your clone and pushing to your fork/branch. Your PR will automatically be updated when commits are pushed to its source branch in your fork, so reviewers will immediately see your updates. When a PR has met the contribution and testing requirements and has been approved by two code reviewers, a code manager will merge the PR. + +.. _pr-template: + +PR Template +------------ + +Here is the template that is provided when developers click "Create pull request": + +.. code-block:: console + + - Update develop to head at ufs-community + + - Use this template to give a detailed message describing the change you want to make to the code. + + - You may delete any sections labeled "optional" and any instructions within . + + - If you are unclear on what should be written here, see https://github.com/wrf-model/WRF/wiki/Making-a-good-pull-request-message for some guidance and review the Code Contributor's Guide at https://github.com/ufs-community/ufs-srweather-app/wiki/Code-Manager's-Guide. + + - Code reviewers will assess the PR based on the criteria laid out in the Code Reviewer's Guide (https://github.com/ufs-community/ufs-srweather-app/wiki/Code-Manager's-Guide). + + - The title of this pull request should be a brief summary (ideally less than 100 characters) of the changes included in this PR. Please also include the branch to which this PR is being issued (e.g., "[develop]: Updated UFS_UTILS hash"). + + - Use the "Preview" tab to see what your PR will look like when you hit "Create pull request" + + + # --- Delete this line and those above before hitting "Create pull request" --- + + ## DESCRIPTION OF CHANGES: + + + ### Type of change + + - [ ] Bug fix (non-breaking change which fixes an issue) + - [ ] New feature (non-breaking change which adds functionality) + - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) + - [ ] This change requires a documentation update + + ## TESTS CONDUCTED: + + + + - [ ] hera.intel + - [ ] orion.intel + - [ ] hercules.intel + - [ ] cheyenne.intel + - [ ] cheyenne.gnu + - [ ] derecho.intel + - [ ] gaea.intel + - [ ] gaeac5.intel + - [ ] jet.intel + - [ ] wcoss2.intel + - [ ] NOAA Cloud (indicate which platform) + - [ ] Jenkins + - [ ] fundamental test suite + - [ ] comprehensive tests (specify *which* if a subset was used) + + ## DEPENDENCIES: + + + ## DOCUMENTATION: + + + ## ISSUE: + + + ## CHECKLIST + + - [ ] My code follows the style guidelines in the Contributor's Guide + - [ ] I have performed a self-review of my own code using the Code Reviewer's Guide + - [ ] I have commented my code, particularly in hard-to-understand areas + - [ ] My changes need updates to the documentation. I have made corresponding changes to the documentation + - [ ] My changes do not require updates to the documentation (explain). + - [ ] My changes generate no new warnings + - [ ] New and existing tests pass with my changes + - [ ] Any dependent changes have been merged and published + + ## LABELS (optional): + + A Code Manager needs to add the following labels to this PR: + - [ ] Work In Progress + - [ ] bug + - [ ] enhancement + - [ ] documentation + - [ ] release + - [ ] high priority + - [ ] run_ci + - [ ] run_we2e_fundamental_tests + - [ ] run_we2e_comprehensive_tests + - [ ] Needs Cheyenne test + - [ ] Needs Jet test + - [ ] Needs Hera test + - [ ] Needs Orion test + - [ ] help wanted + + ## CONTRIBUTORS (optional): + + +.. _pr-template-guidance: + +PR Template Guidance +--------------------- + +**TITLE:** Titles should start with the branch name in brackets and should give code reviewers a clear idea of what the change will do in approximately 5-10 words. Some good examples: + + * [develop] Make thompson_mynn_lam3km ccpp suite available + * [release/public-v2] Add a build_linux_compiler modulefile + * [develop] Fix module loads on Hera + * [develop] Add support for Rocoto with generic LINUX platform + +All of the above examples concisely describe the changes contained in the pull request. The title will not get cut off in emails and web pages. In contrast, here are some made-up (but plausible) examples of BAD pull request titles: + + * Bug fixes (Bug fixes on what part of the code?) + * Changes to surface scheme (What kind of changes? Which surface scheme?) + +**DESCRIPTION OF CHANGES:** The first line of the description should be a single-line "purpose" for this change. Note the type of change (i.e., bug fix, feature/enhancement, text-only). Summarize the problem, proposed solution, and required changes. If this is an enhancement or new feature, describe why the change is important. + +**DOCUMENTATION:** Developers should include documentation on new capabilities and enhancements by updating the appropriate RST documentation files in their fork prior to opening the PR. These documentation updates should be noted in the "Documentation" section of the PR message. If necessary, contributors may submit the RST documentation in a subsequent PR. In these cases, the developers should include any existing documentation in the "Documentation" section of the initial PR message or as a file attachment to the PR. Then, the contributor should open an issue reflecting the need for official RST documentation updates and include the issue number and explanation in the "Documentation" section of the initial PR template. + +.. _tips-best-practices: + +Tips, Best Practices, and Protocols to Follow When Issuing a PR +----------------------------------------------------------------- + +* **Label PR status appropriately.** If the PR is not completely ready to be merged, please add a "Work In Progress" label. Urgent PRs should be marked "high priority." All PRs should have a type label (e.g., "bug," "enhancement"). Labels can be added on the right-hand side of a submitted PR request by clicking on the gear icon beside "Labels" (below the list of reviewers). If users do not have the permissions to add a label to their PR, they should request in their PR description that a code manager add the appropriate labels. +* **Indicate urgency.** If a PR is particularly urgent, this information should be provided in the PR "Description" section, and multiple code management team members should be tagged to draw attention to this PR. After submitting the PR, a "high priority" label should be added to it. +* **Indicate the scope of the PR.** If the PR is extremely minor (e.g., change to the README file), indicate this in the PR message. If it is an extensive PR, the developer should test it on as many platforms as possible and stress the necessity that it be tested on systems for which they do not have access. +* **Clarify in the PR message where the code has been tested.** At a minimum, code should be tested on the platform where code modification has taken place. It should also be tested on machines where code modifications will impact results. If the developer does not have access to these platforms, this should be noted in the PR. +* **Follow separation of concerns.** For example, module loads are only handled in the appropriate modulefiles, Rocoto always sets the work directory, j-jobs make the work directory, and ex-scripts require the work directory to exist. +* **Target subject matter experts (SMEs) among the code management team.** When possible, tag team members who are familiar with the modifications made in the PR so that the code management team can provide effective and streamlined PR reviews and approvals. Developers can tag SMEs by selecting the gear icon next to "Assignees" (under the Reviewers list) and adding the appropriate names. +* **Schedule a live code review** if the PR is exceptionally complex in order to brief members of the code management team on the PR either in-person or through a teleconference. Developers should indicate in the PR message that they are interested in a live code review if they believe that it would be beneficial. + +Merging +======== + +Your PR is ready to merge when: + +#. It has been approved by a required number of ``ufs-srweather-app`` reviewers, including at least one code manager. +#. All conversations have been marked as resolved. +#. All required checks have passed. + +These criteria and their current statuses are detailed in a section at the bottom of your PR's *Conversation* tab. Checks take some time to run, so please be patient. + +In general, the lead code manager will merge the PR when ready. Developers with write permissions should not merge their code themselves unless instructed otherwise by the lead code manager. + +Need Help? +=========== + +See the :ref:`User Support ` section for an overview of user support options. For assistance directly related to a PR, please use comments in the *Conversation* tab of your PR to ask for help with any difficulties you encounter! diff --git a/doc/ContribGuide/documentation.rst b/doc/ContribGuide/documentation.rst new file mode 100644 index 0000000000..9e0bad6bda --- /dev/null +++ b/doc/ContribGuide/documentation.rst @@ -0,0 +1,72 @@ +.. _doc-guidelines: + +Documentation +============= + +.. note:: + + Thank you to the Unified Workflow (UW) team for allowing us to adapt their documentation guidance for use in the SRW App. The original can be viewed in the `uwtools` :uw:`documentation `. + + +Locally Building and Previewing Documentation +--------------------------------------------- + +To locally build the docs: + +#. Install ``sphinx``, ``sphinx-rtd-theme``, and ``sphinxcontrib-bibtex`` on your system if they are not already installed. +#. From the root of your clone: ``cd doc`` +#. Build the docs: ``make doc`` + +The ``make doc`` command will build the documentation under ``doc/build/html``, after which you can preview them in your web browser at the URL: + +.. code-block:: text + + file:///doc/build/html/index.html + +Rerun ``make doc`` and refresh your browser after making and saving changes. + +Viewing Online Documentation +---------------------------- + +Online documentation generation and hosting for the SRW App is provided by :rtd:`Read the Docs<>`. The green *View Docs* button near the upper right of that page links to the official docs for the project. When viewing the documentation, the version selector at the bottom of the navigation column on the left can be used to switch between the latest development code (``develop``), the latest released version (``latest``), and any previously released version. + +Docs are also built and temporarily published when Pull Requests (PRs) targeting the ``develop`` branch are opened. Visit the :rtd:`Builds page` to see recent builds, including those made for PRs. Click a PR-related build marked *Passed*, then the small *View docs* link (**not** the large green *View Docs* button) to see the docs built specifically for that PR. If your PR includes documentation updates, it may be helpful to include the URL of this build in your PR's description so that reviewers can see the rendered HTML docs and not just the modified ``.rst`` files. Note that if commits are pushed to the PR's source branch, Read the Docs will rebuild the PR docs. See the checks section near the bottom of a PR for current status and for another link to the PR docs via the *Details* link. + +.. COMMENT: Technically, docs are built when any PR is opened, regardless of branch. Look into changing. + +Documentation Guidelines +------------------------ + +Please follow these guidelines when contributing to the documentation: + +* Keep formatting consistent across pages. Update all pages when better ideas are discovered. Otherwise, follow the conventions established in existing content. +* Ensure that the ``make doc`` command completes with no errors or warnings. +* If the link-check portion of ``make doc`` reports that a URL is ``permanently`` redirected, update the link in the docs to use the new URL. Non-permanent redirects can be left as-is. +* Do not manually wrap lines in the ``.rst`` files. Insert newlines only as needed to achieve correctly formatted HTML, and let HTML wrap long lines and/or provide a scrollbar. +* Use one blank line between documentation elements (headers, paragraphs, code blocks, etc.) unless additional lines are necessary to achieve correctly formatted HTML. +* Remove all trailing whitespace. +* In general, avoid pronouns like "we" and "you". (Using "we" may be appropriate when synonymous with "The SRW Code Management Team", "The UFS Community", etc., when the context is clear.) Prefer direct, factual statements about what the code does, requires, etc. +* Use the `Oxford Comma `__. +* Follow the :rst:`RST Sections` guidelines, underlining section headings with = characters, subsections with - characters, and subsubsections with ^ characters. If a further level of refinement is needed, use " to underline paragraph headers. +* In [[sub]sub]section titles, capitalize all "principal" words. In practice this usually means all words but articles (a, an, the), logicals (and, etc.), and prepositions (for, of, etc.). Always fully capitalize acronyms (e.g., YAML). +* Never capitalize proper names when their owners do not (e.g., write `"pandas" `__, not "Pandas", even at the start of a sentence) or when referring to a software artifact (e.g., write ``numpy`` when referring to the library, and "NumPy" when referring to the project). +* When referring to YAML constructs, ``block`` refers to an entry whose values is a nested collection of key/value pairs, while ``entry`` is a single key/value pair. +* When using the ``.. code-block::`` directive, align the actual code with the word ``code``. Also, when ``.. code-block::`` directives appear in bulleted or numberd lists, align them with the text following the space to the right of the bullet/number. Include a blank line prior to the coe content. For example: + + .. code-block:: text + + * Lorem ipsum + + .. code-block:: python + + n = 88 + + or + + .. code-block:: text + + #. Lorem ipsum + + .. code-block:: python + + n = 88 \ No newline at end of file diff --git a/doc/ContribGuide/git-submodules.rst b/doc/ContribGuide/git-submodules.rst new file mode 100644 index 0000000000..42e7616cfe --- /dev/null +++ b/doc/ContribGuide/git-submodules.rst @@ -0,0 +1,225 @@ +============================ +Working with Git Submodules +============================ + +.. note:: + + Thank you to Janet Derrico (@jderrico-noaa) [#f1]_ for authoring the summary of Git submodules on which this chapter is based. [#f2]_ It has been adapted slightly for use in the SRW App. + +What Are Git Submodules? +========================= + +Git submodules are pointers to other Git repositories. They enable developers to include external repositories as a subdirectory within their main project. This is particularly useful when a project depends on external libraries or components that are developed and maintained in separate repositories. + +Key Benefits +============= + +* **Version Control:** Submodules link to specific commits in external repositories, ensuring consistency and predictability. Developers can control exactly which version of an external repository their project depends on. +* **Separate Development:** Changes to submodules are tracked separately from the main repository, allowing for independent development of external dependencies. +* **Collaborative Workflows:** Multiple teams can work on different parts of a larger project simultaneously without interference, each with its own repository (e.g. changes to ``ccpp-physics`` can be developed at the same time as changes to ``ufs-weather-model``). + +How Submodules Are Linked +========================== + +Git knows which submodules to check out based on two key pieces of information: the submodule pointer, and the information on where to find that pointer. The pointer is a commit reference---when you add a submodule to your repository, Git doesn't just store the URL; it also records a specific commit hash from that submodule. The commit hash is what Git uses to know which exact state of the submodule to checkout. These commit references are stored in the main repository and are updated whenever a change is committed in the submodule. When you run ``git submodule update``, Git checks out the commit of each submodule according to what is recorded in the main repository. The ``.gitmodules`` file tracks where to find this information, storing the submodule's path within your repository and its corresponding URL. + +If you commit a hash in a submodule but push to a different fork, then Git will add the new submodule hash to the supermodule, which will result in a Git error when trying to recursively check out the supermodule. + +Adding a Submodule +=================== + +You can add a submodule to your repository using ``git submodule add ``. This clones the external repository to the specified path and adds a new entry in a special file named ``.gitmodules``. + +Cloning a Repository with Submodules +===================================== +When cloning a repository that has submodules, use git clone --recursive to ensure that all submodules are also cloned. + +Updating a Submodule +====================== + +To update a submodule, navigate into the submodule directory, check out the desired commit or branch, and then go back to the main repository to commit this change. Here is an example for making a change to ``ccpp-physics``, ``fv3``, and ``ufs-weather-model``. Since ``ccpp-phsyics`` is a submodule of ``fv3atm`` and ``ufs-weather-model``, a change to ``ccpp-physics`` requires PRs to all three repositories. +This method requires two remotes on your local workspace: the authoritative (e.g., ``ufs-community/ufs-weather-model``) and the personal fork you push to (e.g., ``jderrico-noaa/ufs-weather-model``). The steps involved are: + +#. Clone locally +#. Create your working branches +#. Commit your changes +#. Push your working branches to your personal fork +#. Submit PRs from personal fork to authoritative + +Cloning the Authoritative Repository and Adding Your Personal Fork +-------------------------------------------------------------------- + +Clone the authoritative repository to your local workspace: + +.. code-block:: console + + git clone --recursive -b branch-name https://github.com/ufs-community/ufs-weather-model + cd ufs-weather-model + +where ``branch-name`` is the name of the branch you want to clone (usually ``develop``). + +Adding Your Personal Fork as a Remote Repository +-------------------------------------------------- + +.. code-block:: console + + git remote add my-fork + +where ``my-fork`` is the name of your fork. You can name your fork whatever you want as long as you can distinguish it from the authoritative (e.g., janet) https://github.com//ufs-weather-model + +Run: + +.. code-block:: console + + git remote -v + +to show the remote repositories that have been added to your local copy of ``ufs-weather-model``, if should show origin (the authoritative ufs-community repo) and my-fork (your personal fork that you push changes to) +The local repository for ufs-weather-model has been created. This process is repeated for the other submodules (``fv3atm`` and ``ccpp-physics``, where the code will be modified): + +.. code-block:: console + + cd FV3 + git remote add my-fork https://github.com//fv3atm + cd ccpp/physics + git remote add my-fork https://github.com//ccpp-physics + +Create Working Branches +------------------------ + +The next step is to create working branches that will hold your changes until they are merged. From ``ccpp-physics``, navigate up to ``ufs-weather-model``. It is good practice to checkout the main branch (e.g., ``develop``) to ensure that you are working with the latest updates and then create your working branch. You will do this all the way down: + + +Then, navigate from ``ccpp/physics`` back to to ``ufs-weather-model`` and create a new branch to hold your changes: + +.. code-block:: console + + cd ../../.. + git checkout -b working_branch + +This command creates a new branch named ``working_branch``; in practice the branch name should be more descriptive and reflect the development it will be holding. Follow the same process for the Git submodules you will be working in: + +.. code-block:: console + + cd FV3 + git checkout develop + git checkout -b working_branch + cd ccpp/physics + git checkout ufs/dev + git checkout -b working_branch + +Commit Changes and Push Working Branches +------------------------------------------ + +As you make changes to the code, you should commit often. This ensures that all of your development is tracked (so you don't lose anything) and makes it easier to go back to a working version if one of your changes breaks things (it happens!). Commit messages should be descriptive of the changes they contain. + +To push your working branches to your fork from the top down, navigate to the ``ufs-weather model`` directory. Then run: + +.. code-block:: console + + git push -u my-fork working_branch + +The ``-u`` flag here tells Git to set ``my-fork/working_branch`` as the default remote branch for ``working_branch``. After executing this command, you can simply use ``git push`` or ``git pull`` while on ``working_branch``, and Git will automatically know to push or pull from ``my_fork/working_branch``. + +Continue this process with the other submodule repositories: + +.. code-block:: console + + cd FV3 + git push -u my-fork working_branch + cd ccpp/physics + git push -u my-fork working_branch + +All working changes are now in your personal fork. + +Submitting PRs +--------------- + +When working with Git submodules, developers must submit individual pull requests to each repository where changes were made and link them to each other. In this case, developers would submit PRs to ``ufs-weather-model``, ``fv3atm``, and ``ccpp-physics``. There are several steps to this process: opening the PR, updating the submodules, and creating new submodule pointers. Each authoritative repository should have its own PR template that includes space to link to the URLs of related PRs. If for some reason this is not the case, developers should link to the related PRs in the "Description" section of their PR. + +Updating the Submodules +^^^^^^^^^^^^^^^^^^^^^^^^ + +When changes are made to the authoritative repositories while you are developing or while your PR is open, you need to update the PR to include those updates. From your local workspace, navigate to ``ufs-weather-model`` and run: + +.. code-block:: console + + git checkout develop + git pull origin develop + git checkout working_branch + git merge develop + git push -u my-fork working_branch + +This will check out the ``develop`` branch, retrieve the latest updates, then check out the ``working_branch`` and merge the latest changes from ``develop`` into it. After pushing the changes on ``working_branch`` to your personal fork, your PR will update automatically. This process must then be repeated for the other components (e.g., ``fv3`` and ``ccpp-physics``). It is important to check that you are merging the correct branch---for example, the main development branch in ``ufs-community/ccpp-physics`` is ``ufs/dev``, so you would checkout/pull ``ufs/dev`` instead. + +.. note:: + + If you have already pushed ``working_branch`` to ``my-fork`` using the ``-u`` flag, you can omit the flag and fork specification, but it doesn't hurt to use them. + +Add Submodule Pointers +^^^^^^^^^^^^^^^^^^^^^^^ +To create submodule pointers, developers will navigate to the lowest submodule directory (rather than going from the top down) to create pointers linking the submodule to the supermodule. In this example, we are using *ufs-weather-model → fv3 → ccpp-physics*, so developers would start by navigating to ``ccpp-physics``. Once your PR to ``ccpp-physics`` is merged, you then need to update your PRs to ``fv3`` and ``ufs-weather-model`` so that they point to the updated ``ccpp-physics`` submodule. + +First, update the local copy of ``ccpp-physics`` with what was merged to the authoritative (e.g., your changes): + +.. code-block:: console + + git checkout ufs/dev + git pull origin ufs/dev + +Then navigate to ``fv3atm``: + +.. code-block:: console + + cd ../.. + +If you were working with other submodules, you would navigate to submodule above the lowest here. Then create the submodule pointer, commit the change, and push it to your fork of ``fv3atm``: + +.. code-block:: console + + git checkout working_branch + git add ccpp/physics + git commit -m "update submodule pointer for ccpp-physics" + git push -u my-fork working_branch + +Once again, pushing to your personal fork will automatically update the PR that includes ``working_branch``. + +The ``fv3atm`` code managers will then merge your ``fv3atm`` PR, at which point only the ``ufs-weather-model`` PR will require a submodule pointer update. From your local workspace, navigate to the ``fv3`` directory (``ufs-weather-model/FV3``) and update the local copy of ``fv3atm`` with what was just merged into the authoritative: + +.. code-block:: console + + git checkout develop + git pull origin develop + +Then, navigate up to ``ufs-weather model`` directory, check out the working branch, and add the submodule pointer for ``fv3atm``. Commit and push the changes to your personal fork. + +.. code-block:: console + + cd .. + git checkout working_branch + git add FV3 + git commit -m "update submodule pointer for fv3atm" + git push -u my-fork + +The UFS code managers will then test and merge the ``ufs-weather-model`` PR. + +Switching Branches With Submodules +==================================== + +If you are working off a branch that has different versions (or commit references/pointers) of submodules, it is important to synchronize the submodules correctly. From the supermodule, you would switch to your desired branch and then update the submodules. For example, if you want to work on a different branch of the ``ufs-weather-model`` repository: + +.. code-block:: console + + git checkout desired_branch + git submodule update --init --recursive + +Here, ``--init`` initializes any submodules that have not yet been initialized, while ``--recursive`` ensures that all nested submodules (e.g., ``fv3atm``) are updated. If you know there have been upstream changes to a submodule, and you want to incorporate these latest changes, you would go into each submodule directory and pull the changes: + +.. code-block:: console + + cd path/to/submodule + git pull origin + +When working with submodules, it is best practice to always run ``git submodule update --init --recursive`` after switching branches. Changes to submodules need to be committed and pushed separately within their respective repositories (see sections above). + +.. [#f1] of NOAA Global Systems Laboratory (GSL) and Coorperative Institute for Research in Environmental Sciences (CIRES) +.. [#f2] with the assistance of Grant Firl, Joseph Olson, and ChatGPT \ No newline at end of file diff --git a/doc/ContribGuide/index.rst b/doc/ContribGuide/index.rst new file mode 100644 index 0000000000..c4adfc997d --- /dev/null +++ b/doc/ContribGuide/index.rst @@ -0,0 +1,13 @@ +Contributor's Guide +====================== + +.. toctree:: + :maxdepth: 3 + + introduction + contributing + code-configuration-standards + testing + git-submodules + documentation + diff --git a/doc/ContribGuide/introduction.rst b/doc/ContribGuide/introduction.rst new file mode 100644 index 0000000000..687dc2ea25 --- /dev/null +++ b/doc/ContribGuide/introduction.rst @@ -0,0 +1,31 @@ +================= +Introduction +================= + +Background +============ + +Authoritative Branch +---------------------- + +The ``ufs-srweather-app`` repository maintains a main branch for development called ``develop``. The HEAD of ``develop`` reflects the latest development changes. It points to regularly updated hashes for individual subcomponents. Pull requests (PRs) are typically merged to ``develop``. + +The ``develop`` branch is protected by the code management team: + + #. Pull requests for this branch require approval by at least two code reviewers. + #. A code manager should perform at least one of the reviews and the merge, but other contributors are welcome to provide comments/suggestions. + +.. _rcm-team: + +Repository Code Management Team +--------------------------------- + +Scientists and engineers from multiple labs and organizations have volunteered to review pull requests for the ``develop`` branch: + +.. csv-table:: + :file: ../tables/code-managers.csv + :widths: auto + :delim: ; + :header-rows: 1 + + diff --git a/doc/ContribGuide/testing.rst b/doc/ContribGuide/testing.rst new file mode 100644 index 0000000000..b296a3f90a --- /dev/null +++ b/doc/ContribGuide/testing.rst @@ -0,0 +1,70 @@ +.. _pr-testing: + +======== +Testing +======== + +The ``ufs-srweather-app`` repository uses the established workflow end-to-end (WE2E) testing framework (see :ref:`WE2E tests `) to implement two tiers of testing: fundamental and comprehensive. *Fundamental testing* consists of a lightweight set of tests that can be automated and run regularly on each :srw-wiki:`Level 1 ` platform. These tests verify that there are no major, obvious faults in the underlying code when running common combinations of grids, input data, and physics suites. *Comprehensive testing* includes the entire set of WE2E tests and covers a broader range of capabilities, configurations, and components. Eventually, new categories of tests will be added, including regression tests and unit tests. + +Before opening a PR, a minimum set of tests should be run: + + * Developers must run the fundamental test suite manually on at least one supported platform and report on the outcome in the PR template. Developers should test code modifications on as many platforms as possible. + + * To run the fundamental tests manually, run the following command from the ``tests/WE2E`` directory: + + .. code-block:: console + + ./run_WE2E_tests.py -t=fundamental -m=your_machine -a=your_account + + where ``your_machine`` is the Tier-1 machine you are running the tests on, and ``your_account`` is the account you charge your computational resources to. Refer to the :ref:`WE2E Tests ` chapter of the User's Guide for more detail on how to run SRW App tests. + + * Developers will not be required to run tests on *all* supported platforms, but if a failure is pointed out by another reviewer (or by automated testing), then it is expected that the developer will work with reviewers and code managers to ensure that the problem is resolved prior to merging. + + * If the PR impacts functionality contained within comprehensive WE2E tests not included in the fundamental test suite, the developer must run those tests on the PR. + * Any new functionality must be tested explicitly, and any new tests should be described in detail in the PR message. Depending on the impact of this functionality, new tests should be added to the suite of comprehensive WE2E tests, followed by a discussion with code managers on whether they should also be included as fundamental tests. + + * In some cases, it may be possible to modify a current test instead of creating a completely new test. Code developers introducing new capabilities should work with code managers to provide the proper configuration files, data, and other information necessary to create new tests for these capabilities. + + * When the above tests are complete and the PR has been approved by at least one code manager, a code manager will add the ``run_we2e_coverage_tests`` label to initiate fundamental testing on all Level 1 platforms via the Jenkins CI/CD pipeline. + +Testing on Jenkins +=================== + +`Jenkins `__ is an "open source automation server" that automates code testing. For the Jenkins automated testing labels, it should be noted that **ONLY** code managers should apply these labels and only after at least one code manager has given approval to the PR. The PR will not be merged until all Jenkins-based builds and testing have successfully passed. + +The following automated testing labels are available for the SRW App: + + * ``run_we2e_coverage_tests`` + * *Coming Soon:* ``run_we2e_comprehensive_tests`` + +Due to a security issue on Jenkins, where all Jenkins usernames are exposed, access to Jenkins logs through the Jenkins API has been disabled for the public. However, users can visit the `EPIC Health Dashboard `__ and click the *Jenkins Artifacts* tab to access the log files for their PR. On that page, users can identify their PR number, pull the ``we2e_test_logs-{machine}-{compiler}.tgz`` file (where ``{machine}`` is the Tier-1 platform that failed and ``{compiler}`` is the compiler used for the failed test), untar and ungzip the file, and look through the logs from the test that failed. + +Additionally, users can potentially access the directories where the Jenkins tests are run on the various machines so that they can view the tests, monitor progress, and investigate failures. The locations of the experiment directories on the various machines are as follows: + +.. list-table:: + :header-rows: 1 + + * - Tier-1 Platform + - Location of Jenkins experiment directories + * - Derecho + - /glade/derecho/scratch/epicufsrt/jenkins/workspace + * - Gaea + - /lustre/f2/dev/wpo/role.epic/jenkins/workspace/fs-srweather-app_pipeline_PR-#/gaea + * - Gaea C5 + - /lustre/f2/dev/wpo/role.epic/jenkins/workspace/fs-srweather-app_pipeline_PR-#/gaea-c5 + * - Hera (Intel) + - /scratch2/NAGAPE/epic/role.epic/jenkins/workspace/fs-srweather-app_pipeline_PR-#__2/hera + * - Hera (GNU) + - /scratch2/NAGAPE/epic/role.epic/jenkins/workspace/fs-srweather-app_pipeline_PR-#/hera + * - Hercules + - /work/noaa/epic/role-epic/jenkins/workspace/fs-srweather-app_pipeline_PR-#/hercules + * - Jet + - /lfs1/NAGAPE/epic/role.epic/jenkins/workspace/fs-srweather-app_pipeline_PR-#/jet + * - Orion + - /work/noaa/epic/role-epic/jenkins/workspace/fs-srweather-app_pipeline_PR-#/orion + +where ``#`` is the PR number. + +If the Jenkins tests fail, then the developer will need to make the necessary corrections to their PR. Unfortunately, removing and adding the label back will not kick off the Jenkins test again. Instead, the job will need to be manually re-run through Jenkins (by a member of the EPIC team). + + diff --git a/doc/INSTALL b/doc/INSTALL index e53044f6ad..53dc159bbd 100644 --- a/doc/INSTALL +++ b/doc/INSTALL @@ -12,7 +12,7 @@ git clone https://github.com/ufs-community/ufs-srweather-app.git cd ufs-srweather-app/ ./manage_externals/checkout_externals -# We can build ufs-sreweather-app binaries in two ways. +# We can build ufs-srweather-app binaries in two ways. # Method 1 # ======== diff --git a/doc/UsersGuide/Makefile b/doc/Makefile similarity index 84% rename from doc/UsersGuide/Makefile rename to doc/Makefile index 84c77bbfa2..c91f2f147b 100644 --- a/doc/UsersGuide/Makefile +++ b/doc/Makefile @@ -2,7 +2,7 @@ SPHINXOPTS = -a -n #-W SPHINXBUILD = sphinx-build -SOURCEDIR = source +SOURCEDIR = . BUILDDIR = build LINKCHECKDIR = $(BUILDDIR)/linkcheck @@ -12,12 +12,13 @@ LINKCHECKDIR = $(BUILDDIR)/linkcheck help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) -docs: +doc: + make clean $(MAKE) linkcheck $(MAKE) html linkcheck: - make clean && $(SPHINXBUILD) -b linkcheck $(SPHINXOPTS) $(SOURCEDIR) $(LINKCHECKDIR) + $(SPHINXBUILD) -b linkcheck $(SPHINXOPTS) $(SOURCEDIR) $(LINKCHECKDIR) # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). diff --git a/doc/UsersGuide/README b/doc/README similarity index 100% rename from doc/UsersGuide/README rename to doc/README diff --git a/doc/UsersGuide/source/BackgroundInfo/CCPPUpdates.rst b/doc/UsersGuide/BackgroundInfo/CCPPUpdates.rst similarity index 100% rename from doc/UsersGuide/source/BackgroundInfo/CCPPUpdates.rst rename to doc/UsersGuide/BackgroundInfo/CCPPUpdates.rst diff --git a/doc/UsersGuide/source/BackgroundInfo/Components.rst b/doc/UsersGuide/BackgroundInfo/Components.rst similarity index 99% rename from doc/UsersGuide/source/BackgroundInfo/Components.rst rename to doc/UsersGuide/BackgroundInfo/Components.rst index d861304502..1ba9349d8d 100644 --- a/doc/UsersGuide/source/BackgroundInfo/Components.rst +++ b/doc/UsersGuide/BackgroundInfo/Components.rst @@ -89,7 +89,7 @@ For more information on NEXUS, visit the GitHub repository at https://github.com Unified Workflow Tools ======================== -The Unified Workflow (UW) is a set of tools intended to unify the workflow for various UFS applications under one framework. The UW toolkit currently includes templater and configuration (config) tools, which have been incorporated into the SRW App workflow and will soon be incorporated into other UFS repositories. Additional tools are under development. More details about UW tools can be found in the `uwtools `__ GitHub repository and in the :doc:`UW Documentation `. +The Unified Workflow (UW) is a set of tools intended to unify the workflow for various UFS applications under one framework. The UW toolkit currently includes templater and configuration (config) tools, which have been incorporated into the SRW App workflow and will soon be incorporated into other UFS repositories. Additional tools are under development. More details about UW tools can be found in the `uwtools `__ GitHub repository and in the :uw:`UW Documentation <>`. Build System and Workflow ========================= diff --git a/doc/UsersGuide/source/BackgroundInfo/Introduction.rst b/doc/UsersGuide/BackgroundInfo/Introduction.rst similarity index 78% rename from doc/UsersGuide/source/BackgroundInfo/Introduction.rst rename to doc/UsersGuide/BackgroundInfo/Introduction.rst index 4c6379e295..f1a384e025 100644 --- a/doc/UsersGuide/source/BackgroundInfo/Introduction.rst +++ b/doc/UsersGuide/BackgroundInfo/Introduction.rst @@ -11,28 +11,30 @@ The UFS includes `multiple applications `__) and support for the ``RRFS_NA_13km`` predefined grid - * Addition of ``FV3_GFS_v17_p8`` physics suite (`PR #574 `__) + * Addition of the supported ``FV3_RAP`` physics suite (:srw-repo:`PR #811 `) and support for the ``RRFS_NA_13km`` predefined grid + * Addition of ``FV3_GFS_v17_p8`` physics suite (:srw-repo:`PR #574 `) * Updates to :term:`CCPP` that target the top of the ``main`` branch (which is ahead of CCPP v6.0.0). See :ref:`this page ` for a detailed summary of updates that came in ahead of the v2.2.0 release. - * Expansion of :srw-wiki:`Level 1 platforms ` to include Derecho, Hercules, and Gaea C5 (PRs `#894 `__, `#898 `__, `#911 `__) - * Transition to spack-stack modulefiles for most supported platforms to align with the UFS WM shift to spack-stack (PRs `#913 `__ and `#941 `__) - * Overhaul of the WE2E testing suite (see, e.g., PRs `#686 `__, `#732 `__, `#864 `__, `#871 `__) - * Improvements to the CI/CD automated testing pipeline (see, e.g., PRs `#707 `__ and `#847 `__) - * Incorporation of additional METplus verification capabilities (PRs `#552 `__, `#614 `__, `#757 `__, `#853 `__) - * Integration of the Unified Workflow's templater tool (`PR #793 `__) - * Ability to create a user-defined custom workflow (`PR #676 `__) - * Option to use a custom vertical coordinate file with different distribution of vertical layers (`PR #813 `__) and :ref:`documentation on how to use this feature ` (`PR #888 `__) - * Incorporation of plotting tasks into the workflow (PR `#482 `__); addition of ability to plot on both CONUS and smaller regional grid (`PR #560 `__) - * Addition of a sample verification case (`PR #500 `__) with :ref:`documentation ` - * A new :ref:`tutorial chapter ` in the documentation (`PR #584 `__) - * Incorporation of `UFS Case Studies `__ within the WE2E framework (PRs `#736 `__ and `#822 `__) - * Air Quality Modeling (AQM) capabilities (unsupported but available; see `PR #613 `__) + * Expansion of :srw-wiki:`Level 1 platforms ` to include Derecho, Hercules, and Gaea C5 (PRs :srw-repo:`#894 `, :srw-repo:`#898 `, :srw-repo:`#911 `) + * Transition to spack-stack modulefiles for most supported platforms to align with the UFS WM shift to spack-stack (PRs :srw-repo:`#913 ` and :srw-repo:`#941 `) + * Overhaul of the WE2E testing suite (see, e.g., PRs :srw-repo:`#686 `, :srw-repo:`#732 `, :srw-repo:`#864 `, :srw-repo:`#871 `) + * Improvements to the CI/CD automated testing pipeline (see, e.g., PRs :pull/707>` and :srw-repo:`#847 `) + * Incorporation of additional METplus verification capabilities (PRs :srw-repo:`#552 `, :srw-repo:`#614 `, :srw-repo:`#757 `, :srw-repo:`#853 `) + * Integration of the Unified Workflow's templater tool (:srw-repo:`PR #793 `) + * Ability to create a user-defined custom workflow (:srw-repo:`PR #676 `) + * Option to use a custom vertical coordinate file with different distribution of vertical layers (:srw-repo:`PR #813 `) and :ref:`documentation on how to use this feature ` (:srw-repo:`PR #888 `) + * Incorporation of plotting tasks into the workflow (PR :srw-repo:`#482 `); addition of ability to plot on both CONUS and smaller regional grid (:srw-repo:`PR #560 `) + * Addition of a sample verification case (:srw-repo:`PR #500 `) with :ref:`documentation ` + * A new :ref:`tutorial chapter ` in the documentation (:srw-repo:`PR #584 `) + * Incorporation of `UFS Case Studies `__ within the WE2E framework (PRs :srw-repo:`#736 ` and :srw-repo:`#822 `) + * Air Quality Modeling (AQM) capabilities (unsupported but available; see :srw-repo:`PR #613 `) * Miscellaneous documentation updates to reflect the changes above The SRW App |latestr| citation is as follows and should be used when presenting results based on research conducted with the App: UFS Development Team. (2023, Oct. 31). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v2.2.0). Zenodo. https://doi.org/10.5281/zenodo.10015544 +.. _ug-organization: + User's Guide Organization ============================ @@ -80,6 +82,7 @@ Reference Information * :numref:`Section %s: FAQ ` answers users' frequently asked questions. * :numref:`Section %s: Glossary ` defines important terms related to the SRW App. +.. _doc-conventions: SRW App Documentation Conventions =================================== @@ -96,6 +99,8 @@ Code that includes angle brackets (e.g., ``build__``) indica File or directory paths that begin with ``/path/to/`` should be replaced with the actual path on the user's system. For example, ``/path/to/modulefiles`` might be replaced by ``/Users/Jane.Smith/ufs-srweather-app/modulefiles``. +.. _component-docs: + Component Documentation ========================= @@ -142,8 +147,10 @@ A list of available component documentation is shown in :numref:`Table %s `__, since many "bugs" do not require a code change/fix --- instead, the user may be unfamiliar with the system and/or may have misunderstood some component of the system or the instructions, which is causing the problem. Asking for assistance in a `GitHub Discussion `__ post can help clarify whether there is a simple adjustment to fix the problem or whether there is a genuine bug in the code. Users are also encouraged to search `open issues `__ to see if their bug has already been identified. If there is a genuine bug, and there is no open issue to address it, users can report the bug by filing a `GitHub Issue `__. +If users (especially new users) believe they have identified a bug in the system, it is recommended that they first ask about the problem in :srw-repo:`GitHub Discussions `, since many "bugs" do not require a code change/fix --- instead, the user may be unfamiliar with the system and/or may have misunderstood some component of the system or the instructions, which is causing the problem. Asking for assistance in a :srw-repo:`GitHub Discussion ` post can help clarify whether there is a simple adjustment to fix the problem or whether there is a genuine bug in the code. Users are also encouraged to search :srw-repo:`open issues ` to see if their bug has already been identified. If there is a genuine bug, and there is no open issue to address it, users can report the bug by filing a :srw-repo:`GitHub Issue `. Feature Requests and Enhancements ----------------------------------- @@ -178,6 +185,8 @@ utilities, model code, and infrastructure. As described above, users can post is Contributions to the `ufs-srweather-app `__ repository should follow the guidelines contained in the :srw-wiki:`SRW App Contributor's Guide `. Additionally, users can file issues in component repositories for contributions that directly concern those repositories. For code to be accepted into a component repository, users must follow the code management rules of that component's authoritative repository. These rules are usually outlined in the component's User's Guide (see :numref:`Table %s `) or GitHub wiki for each respective repository (see :numref:`Table %s `). +.. _future-direction: + Future Direction ================= @@ -189,4 +198,4 @@ Users can expect to see incremental improvements and additional capabilities in * Incorporation of additional `Unified Workflow `__ tools. -.. bibliography:: ../references.bib +.. bibliography:: ../../references.bib diff --git a/doc/UsersGuide/source/BackgroundInfo/TechnicalOverview.rst b/doc/UsersGuide/BackgroundInfo/TechnicalOverview.rst similarity index 94% rename from doc/UsersGuide/source/BackgroundInfo/TechnicalOverview.rst rename to doc/UsersGuide/BackgroundInfo/TechnicalOverview.rst index b2a1819670..52365a86e5 100644 --- a/doc/UsersGuide/source/BackgroundInfo/TechnicalOverview.rst +++ b/doc/UsersGuide/BackgroundInfo/TechnicalOverview.rst @@ -138,7 +138,7 @@ The UFS Weather Model contains a number of sub-repositories, which are documente Repository Structure ---------------------- -The ``ufs-srweather-app`` :term:`umbrella repository` is an NCO-compliant repository. Its structure follows the standards laid out in :term:`NCEP` Central Operations (NCO) WCOSS `Implementation Standards `__. This structure is implemented using the ``local_path`` settings contained within the ``Externals.cfg`` file. After ``manage_externals/checkout_externals`` is run (see :numref:`Section %s `), the specific GitHub repositories described in :numref:`Table %s ` are cloned into the target subdirectories shown below. Directories that will be created as part of the build process appear in parentheses and will not be visible until after the build is complete. Some directories have been removed for brevity. +The ``ufs-srweather-app`` :term:`umbrella repository` is an NCO-compliant repository. Its structure follows the standards laid out in :term:`NCEP` Central Operations (NCO) WCOSS :nco:`Implementation Standards `. This structure is implemented using the ``local_path`` settings contained within the ``Externals.cfg`` file. After ``manage_externals/checkout_externals`` is run (see :numref:`Section %s `), the specific GitHub repositories described in :numref:`Table %s ` are cloned into the target subdirectories shown below. Directories that will be created as part of the build process appear in parentheses and will not be visible until after the build is complete. Some directories have been removed for brevity. .. code-block:: console @@ -193,7 +193,7 @@ The ``ufs-srweather-app`` :term:`umbrella repository` is an NCO-compliant reposi SRW App SubDirectories ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:numref:`Table %s ` describes the contents of the most important SRW App subdirectories. :numref:`Table %s ` provides a more comprehensive explanation of the ``ufs-srweather-app`` files and subdirectories. Users can reference the `NCO Implementation Standards `__ (p. 19) for additional details on repository structure in NCO-compliant repositories. +:numref:`Table %s ` describes the contents of the most important SRW App subdirectories. :numref:`Table %s ` provides a more comprehensive explanation of the ``ufs-srweather-app`` files and subdirectories. Users can reference the :nco:`NCO Implementation Standards ` (p. 19) for additional details on repository structure in NCO-compliant repositories. .. _Subdirectories: diff --git a/doc/UsersGuide/source/BackgroundInfo/index.rst b/doc/UsersGuide/BackgroundInfo/index.rst similarity index 100% rename from doc/UsersGuide/source/BackgroundInfo/index.rst rename to doc/UsersGuide/BackgroundInfo/index.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/AQM.rst b/doc/UsersGuide/BuildingRunningTesting/AQM.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/AQM.rst rename to doc/UsersGuide/BuildingRunningTesting/AQM.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst b/doc/UsersGuide/BuildingRunningTesting/BuildSRW.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst rename to doc/UsersGuide/BuildingRunningTesting/BuildSRW.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst rename to doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/DefaultVarsTable.rst b/doc/UsersGuide/BuildingRunningTesting/DefaultVarsTable.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/DefaultVarsTable.rst rename to doc/UsersGuide/BuildingRunningTesting/DefaultVarsTable.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/Quickstart.rst b/doc/UsersGuide/BuildingRunningTesting/Quickstart.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/Quickstart.rst rename to doc/UsersGuide/BuildingRunningTesting/Quickstart.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/RunSRW.rst b/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/RunSRW.rst rename to doc/UsersGuide/BuildingRunningTesting/RunSRW.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst rename to doc/UsersGuide/BuildingRunningTesting/Tutorial.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/VXCases.rst b/doc/UsersGuide/BuildingRunningTesting/VXCases.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/VXCases.rst rename to doc/UsersGuide/BuildingRunningTesting/VXCases.rst diff --git a/doc/UsersGuide/source/BuildingRunningTesting/WE2Etests.rst b/doc/UsersGuide/BuildingRunningTesting/WE2Etests.rst similarity index 99% rename from doc/UsersGuide/source/BuildingRunningTesting/WE2Etests.rst rename to doc/UsersGuide/BuildingRunningTesting/WE2Etests.rst index 031038d1d7..4fca53b575 100644 --- a/doc/UsersGuide/source/BuildingRunningTesting/WE2Etests.rst +++ b/doc/UsersGuide/BuildingRunningTesting/WE2Etests.rst @@ -32,7 +32,7 @@ WE2E Test Categories WE2E tests are grouped into two categories that are of interest to code developers: ``fundamental`` and ``comprehensive`` tests. "Fundamental" tests are a lightweight but wide-reaching set of tests designed to function as a cheap "`smoke test `__" for changes to the UFS SRW App. The fundamental suite of tests runs common combinations of workflow tasks, physical domains, input data, physics suites, etc. The comprehensive suite of tests covers a broader range of combinations of capabilities, configurations, and components, ideally including all capabilities that *can* be run on a given platform. Because some capabilities are not available on all platforms (e.g., retrieving data directly from NOAA HPSS), the suite of comprehensive tests varies from machine to machine. -The list of fundamental and comprehensive tests can be viewed in the ``ufs-srweather-app/tests/WE2E/machine_suites/`` directory, and the tests are described in more detail in :doc:`this table <../tables/Tests>`. +The list of fundamental and comprehensive tests can be viewed in the ``ufs-srweather-app/tests/WE2E/machine_suites/`` directory, and the tests are described in more detail in :doc:`this table <../../tables/Tests>`. .. note:: @@ -169,7 +169,7 @@ The script to run the WE2E tests is named ``run_WE2E_tests.py`` and is located i .. note:: - The full list of WE2E tests is extensive, and some larger, high-resolution tests are computationally expensive. Estimates of walltime and core-hour cost for each test are provided in :doc:`this table <../tables/Tests>`. + The full list of WE2E tests is extensive, and some larger, high-resolution tests are computationally expensive. Estimates of walltime and core-hour cost for each test are provided in :doc:`this table <../../tables/Tests>`. Using the Test Script ---------------------- diff --git a/doc/UsersGuide/source/BuildingRunningTesting/index.rst b/doc/UsersGuide/BuildingRunningTesting/index.rst similarity index 100% rename from doc/UsersGuide/source/BuildingRunningTesting/index.rst rename to doc/UsersGuide/BuildingRunningTesting/index.rst diff --git a/doc/UsersGuide/source/CustomizingTheWorkflow/ConfigWorkflow.rst b/doc/UsersGuide/CustomizingTheWorkflow/ConfigWorkflow.rst similarity index 99% rename from doc/UsersGuide/source/CustomizingTheWorkflow/ConfigWorkflow.rst rename to doc/UsersGuide/CustomizingTheWorkflow/ConfigWorkflow.rst index 0c8ed8e951..10a0bcc4eb 100644 --- a/doc/UsersGuide/source/CustomizingTheWorkflow/ConfigWorkflow.rst +++ b/doc/UsersGuide/CustomizingTheWorkflow/ConfigWorkflow.rst @@ -20,7 +20,7 @@ USER Configuration Parameters If non-default parameters are selected for the variables in this section, they should be added to the ``user:`` section of the ``config.yaml`` file. ``RUN_ENVIR``: (Default: "nco") - This variable determines the workflow mode. The user can choose between two options: "nco" and "community". The "nco" mode uses a directory structure that mimics what is used in operations at NOAA/NCEP Central Operations (NCO) and at the NOAA/NCEP/Environmental Modeling Center (EMC), which works with NCO on pre-implementation testing. Specifics of the conventions used in "nco" mode can be found in the following `WCOSS Implementation Standards `__ document: + This variable determines the workflow mode. The user can choose between two options: "nco" and "community". The "nco" mode uses a directory structure that mimics what is used in operations at NOAA/NCEP Central Operations (NCO) and at the NOAA/NCEP/Environmental Modeling Center (EMC), which works with NCO on pre-implementation testing. Specifics of the conventions used in "nco" mode can be found in the following :nco:`WCOSS Implementation Standards <>` document: | NCEP Central Operations | WCOSS Implementation Standards @@ -699,7 +699,7 @@ A standard set of environment variables has been established for *nco* mode to s Only *community* mode is fully supported for releases. *nco* mode is used by those at the Environmental Modeling Center (EMC) and Global Systems Laboratory (GSL) who are working on pre-implementation operational testing. Other users should run the SRW App in *community* mode. ``envir_default, NET_default, model_ver_default, RUN_default``: - Standard environment variables defined in the NCEP Central Operations WCOSS Implementation Standards document. These variables are used in forming the path to various directories containing input, output, and workflow files. The variables are defined in the `WCOSS Implementation Standards `__ document (pp. 4-5) as follows: + Standard environment variables defined in the NCEP Central Operations WCOSS Implementation Standards document. These variables are used in forming the path to various directories containing input, output, and workflow files. The variables are defined in the :nco:`WCOSS Implementation Standards ` document (pp. 4-5) as follows: ``envir_default``: (Default: "para") Set to "test" during the initial testing phase, "para" when running in parallel (on a schedule), and "prod" in production. @@ -1349,7 +1349,7 @@ PLOT_ALLVARS Configuration Parameters Typically, the following parameters must be set explicitly by the user in the ``task_plot_allvars:`` section of the configuration file (``config.yaml``) when executing the plotting tasks. ``COMOUT_REF``: (Default: "") - Path to the reference experiment's COMOUT directory. This is the directory where the GRIB2 files from post-processing are located. In *community* mode (i.e., when ``RUN_ENVIR: "community"``), this directory will correspond to the location in the experiment directory where the post-processed output can be found (e.g., ``$EXPTDIR/$DATE_FIRST_CYCL/postprd``). In *nco* mode, this directory should be set to the location of the ``COMOUT`` directory and end with ``$PDY/$cyc``. For more detail on *nco* standards and directory naming conventions, see `WCOSS Implementation Standards `__ (particularly pp. 4-5). + Path to the reference experiment's COMOUT directory. This is the directory where the GRIB2 files from post-processing are located. In *community* mode (i.e., when ``RUN_ENVIR: "community"``), this directory will correspond to the location in the experiment directory where the post-processed output can be found (e.g., ``$EXPTDIR/$DATE_FIRST_CYCL/postprd``). In *nco* mode, this directory should be set to the location of the ``COMOUT`` directory and end with ``$PDY/$cyc``. For more detail on *nco* standards and directory naming conventions, see :nco:`WCOSS Implementation Standards ` (particularly pp. 4-5). ``PLOT_FCST_START``: (Default: 0) The starting forecast hour for the plotting task. For example, if a forecast starts at 18h/18z, this is considered the 0th forecast hour, so "starting forecast hour" should be 0, not 18. If a forecast starts at 18h/18z, but the user only wants plots from the 6th forecast hour on, "starting forecast hour" should be 6. diff --git a/doc/UsersGuide/source/CustomizingTheWorkflow/DefineWorkflow.rst b/doc/UsersGuide/CustomizingTheWorkflow/DefineWorkflow.rst similarity index 100% rename from doc/UsersGuide/source/CustomizingTheWorkflow/DefineWorkflow.rst rename to doc/UsersGuide/CustomizingTheWorkflow/DefineWorkflow.rst diff --git a/doc/UsersGuide/source/CustomizingTheWorkflow/InputOutputFiles.rst b/doc/UsersGuide/CustomizingTheWorkflow/InputOutputFiles.rst similarity index 98% rename from doc/UsersGuide/source/CustomizingTheWorkflow/InputOutputFiles.rst rename to doc/UsersGuide/CustomizingTheWorkflow/InputOutputFiles.rst index d8266e74e8..128b080655 100644 --- a/doc/UsersGuide/source/CustomizingTheWorkflow/InputOutputFiles.rst +++ b/doc/UsersGuide/CustomizingTheWorkflow/InputOutputFiles.rst @@ -168,8 +168,8 @@ If users wish to modify the fields or levels that are output from the UPP, they This process requires advanced knowledge of which fields can be output for the UFS Weather Model. UPP Product Output Tables for the UFS SRW LAM Grid: - * :doc:`3D Native Hybrid Level Fields <../tables/SRW_NATLEV_table>` - * :doc:`3D Pressure Level Fields <../tables/SRW_PRSLEV_table>` + * :doc:`3D Native Hybrid Level Fields <../../tables/SRW_NATLEV_table>` + * :doc:`3D Pressure Level Fields <../../tables/SRW_PRSLEV_table>` Use the instructions in the `UPP User's Guide `__ to make modifications to the ``fv3lam.xml`` file and to remake the flat text file, called ``postxconfig-NT-fv3lam.txt`` (default), that the UPP reads. @@ -364,11 +364,6 @@ Google Cloud: * HRRR: https://console.cloud.google.com/marketplace/product/noaa-public/hrrr -FTP Data Repository (data for SRW Release v1.0.0 & v1.0.1): - -* https://ftp.emc.ncep.noaa.gov/EIB/UFS/SRW/v1p0/fix/ -* https://ftp.emc.ncep.noaa.gov/EIB/UFS/SRW/v1p0/simple_test_case/ - Others: * University of Utah HRRR archive: https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/hrrr_download.cgi diff --git a/doc/UsersGuide/source/CustomizingTheWorkflow/LAMGrids.rst b/doc/UsersGuide/CustomizingTheWorkflow/LAMGrids.rst similarity index 100% rename from doc/UsersGuide/source/CustomizingTheWorkflow/LAMGrids.rst rename to doc/UsersGuide/CustomizingTheWorkflow/LAMGrids.rst diff --git a/doc/UsersGuide/source/CustomizingTheWorkflow/TemplateVars.rst b/doc/UsersGuide/CustomizingTheWorkflow/TemplateVars.rst similarity index 100% rename from doc/UsersGuide/source/CustomizingTheWorkflow/TemplateVars.rst rename to doc/UsersGuide/CustomizingTheWorkflow/TemplateVars.rst diff --git a/doc/UsersGuide/source/CustomizingTheWorkflow/index.rst b/doc/UsersGuide/CustomizingTheWorkflow/index.rst similarity index 100% rename from doc/UsersGuide/source/CustomizingTheWorkflow/index.rst rename to doc/UsersGuide/CustomizingTheWorkflow/index.rst diff --git a/doc/UsersGuide/source/Reference/FAQ.rst b/doc/UsersGuide/Reference/FAQ.rst similarity index 100% rename from doc/UsersGuide/source/Reference/FAQ.rst rename to doc/UsersGuide/Reference/FAQ.rst diff --git a/doc/UsersGuide/source/Reference/Glossary.rst b/doc/UsersGuide/Reference/Glossary.rst similarity index 98% rename from doc/UsersGuide/source/Reference/Glossary.rst rename to doc/UsersGuide/Reference/Glossary.rst index dc1f810306..90f9c8ab89 100644 --- a/doc/UsersGuide/source/Reference/Glossary.rst +++ b/doc/UsersGuide/Reference/Glossary.rst @@ -90,7 +90,7 @@ Glossary `Earth System Modeling Framework `__. The ESMF defines itself as “a suite of software tools for developing high-performance, multi-component Earth science modeling applications.” ex-scripts - Scripting layer (contained in ``ufs-srweather-app/scripts/``) that should be called by a :term:`J-job ` for each workflow componentto run a specific task or sub-task in the workflow. The different scripting layers are described in detail in the `NCO Implementation Standards document `__ + Scripting layer (contained in ``ufs-srweather-app/scripts/``) that should be called by a :term:`J-job ` for each workflow componentto run a specific task or sub-task in the workflow. The different scripting layers are described in detail in the :nco:`NCO Implementation Standards document ` FV3 The Finite-Volume Cubed-Sphere :term:`dynamical core` (dycore). Developed at NOAA's `Geophysical @@ -133,7 +133,7 @@ Glossary Initial conditions J-jobs - Scripting layer (contained in ``ufs-srweather-app/jobs/``) that should be directly called for each workflow component (either on the command line or by the workflow manager) to run a specific task in the workflow. The different scripting layers are described in detail in the `NCO Implementation Standards document `__ + Scripting layer (contained in ``ufs-srweather-app/jobs/``) that should be directly called for each workflow component (either on the command line or by the workflow manager) to run a specific task in the workflow. The different scripting layers are described in detail in the :nco:`NCO Implementation Standards document ` JEDI The Joint Effort for Data assimilation Integration (`JEDI `__) is a unified and versatile data assimilation (DA) system for Earth System Prediction. It aims to enable efficient research and accelerated transition from research to operations by providing a framework that takes into account all components of the Earth system in a consistent manner. The JEDI software package can run on a variety of platforms and for a variety of purposes, and it is designed to readily accommodate new atmospheric and oceanic models and new observation systems. The `JEDI User's Guide `__ contains extensive information on the software. diff --git a/doc/UsersGuide/source/Reference/RocotoInfo.rst b/doc/UsersGuide/Reference/RocotoInfo.rst similarity index 100% rename from doc/UsersGuide/source/Reference/RocotoInfo.rst rename to doc/UsersGuide/Reference/RocotoInfo.rst diff --git a/doc/UsersGuide/source/Reference/index.rst b/doc/UsersGuide/Reference/index.rst similarity index 100% rename from doc/UsersGuide/source/Reference/index.rst rename to doc/UsersGuide/Reference/index.rst diff --git a/doc/UsersGuide/source/SSHIntro.rst b/doc/UsersGuide/SSHIntro.rst similarity index 100% rename from doc/UsersGuide/source/SSHIntro.rst rename to doc/UsersGuide/SSHIntro.rst diff --git a/doc/UsersGuide/index.rst b/doc/UsersGuide/index.rst new file mode 100644 index 0000000000..58c6fe6089 --- /dev/null +++ b/doc/UsersGuide/index.rst @@ -0,0 +1,10 @@ +User's Guide +============== + +.. toctree:: + :maxdepth: 3 + + BackgroundInfo/index + BuildingRunningTesting/index + CustomizingTheWorkflow/index + Reference/index diff --git a/doc/UsersGuide/source/_static/theme_overrides.css b/doc/UsersGuide/source/_static/theme_overrides.css deleted file mode 100644 index a9672944ba..0000000000 --- a/doc/UsersGuide/source/_static/theme_overrides.css +++ /dev/null @@ -1,24 +0,0 @@ -/* override table width restrictions */ -@media screen and (min-width: 767px) { - - .wy-table-responsive table td { - /* !important prevents the common CSS stylesheets from overriding - this as on RTD they are loaded after this stylesheet */ - white-space: normal !important; - } - - .wy-nav-content { - max-width: 100% !important; - } - - /* .wy-table-responsive { */ - /* overflow: visible !important; */ - /* } */ - -} - -/* Darken navbar blue background for contrast with logo */ -.wy-side-nav-search, .wy-nav-top { - background: #2779B0; -} - diff --git a/doc/UsersGuide/source/index.rst b/doc/UsersGuide/source/index.rst deleted file mode 100644 index 8f975b7cd1..0000000000 --- a/doc/UsersGuide/source/index.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. UFS SR Weather App Users Guide, created by - sphinx-quickstart on Tue Feb 12 08:48:32 2019. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - -UFS Short-Range Weather App User's Guide (|version|) -===================================================== - -.. toctree:: - :numbered: - :maxdepth: 3 - - BackgroundInfo/index - BuildingRunningTesting/index - CustomizingTheWorkflow/index - Reference/index diff --git a/doc/UsersGuide/source/_static/custom.css b/doc/_static/custom.css similarity index 100% rename from doc/UsersGuide/source/_static/custom.css rename to doc/_static/custom.css diff --git a/doc/_static/theme_overrides.css b/doc/_static/theme_overrides.css new file mode 100644 index 0000000000..a4e5cab82f --- /dev/null +++ b/doc/_static/theme_overrides.css @@ -0,0 +1,26 @@ +.wy-table-responsive table td { + /* !important prevents the common CSS stylesheets from overriding + this as on RTD they are loaded after this stylesheet */ + white-space: normal !important; +} + +.wy-nav-content { + max-width: 100% !important; +} + +.wy-table-responsive { + overflow: visible !important; +} + +/* Darken background for contrast with logo */ +.wy-side-nav-search, .wy-nav-top { + background: #2779B0; +} + +/* Darken navbar blue background for contrast with logo */ +.wy-side-nav-search, .wy-nav-top { + background: #2779B0; +} + + + diff --git a/doc/UsersGuide/source/_templates/.gitignore b/doc/_templates/.gitignore similarity index 100% rename from doc/UsersGuide/source/_templates/.gitignore rename to doc/_templates/.gitignore diff --git a/doc/UsersGuide/source/conf.py b/doc/conf.py similarity index 85% rename from doc/UsersGuide/source/conf.py rename to doc/conf.py index cc348f18da..6b0f461ba8 100644 --- a/doc/UsersGuide/source/conf.py +++ b/doc/conf.py @@ -14,7 +14,8 @@ # import os import sys -sys.path.insert(0, os.path.abspath('.')) +import sphinx +from sphinx.util import logging @@ -32,37 +33,10 @@ numfig = True -# Avoid a 403 Forbidden error when accessing certain links (e.g., noaa.gov) -user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" - -# Ignore working links that cause a linkcheck 403 error. -linkcheck_ignore = [r'https://www\.intel\.com/content/www/us/en/docs/cpp\-compiler/developer\-guide\-reference/2021\-10/thread\-affinity\-interface\.html', - r'https://www\.intel\.com/content/www/us/en/developer/tools/oneapi/hpc\-toolkit\-download\.html', - ] - -# Ignore anchor tags for SRW App data bucket. Shows Not Found even when they exist. -linkcheck_anchors_ignore = [r"current_srw_release_data/", - r"input_model_data/.*", - r"fix.*", - r"sample_cases/.*", - ] - -linkcheck_allowed_redirects = {r"https://github\.com/ufs-community/ufs-srweather-app/wiki/.*": r"https://raw\.githubusercontent\.com/wiki/ufs-community/ufs-srweather-app/.*", - r"https://github\.com/ufs-community/ufs-srweather-app/issues/new/choose": r"https://github\.com/login", - r"https://doi\.org/.*/zenodo\..*": r"https://zenodo\.org/records/.*", - r"https://doi\.org/.*": r"https://gmd\.copernicus\.org/.*", - r"https://rdhpcs\-common\-docs\.rdhpcs\.noaa\.gov/wiki/index\.php/Transferring\_Data": - r"https://sso\.noaa\.gov\:443/openam/SSORedirect/metaAlias/noaa\-online/idp\?SAMLRequest\=.*" - } # -- General configuration --------------------------------------------------- -# If your documentation needs a minimal Sphinx version, state it here. -# needs_sphinx = '1.0' - -# Add any Sphinx extension module names here, as strings. They can be -# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom -# ones. +# Sphinx extension module names: extensions = [ 'sphinx_rtd_theme', 'sphinx.ext.autodoc', @@ -78,7 +52,6 @@ ] bibtex_bibfiles = ['references.bib'] -#bibtex_bibfiles = ['refs.bib'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -99,7 +72,8 @@ # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. -exclude_patterns = [] +exclude_patterns = ['_build', + '.DS_Store',] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' @@ -117,6 +91,37 @@ .. |data| replace:: develop """ +# Linkcheck options + +# Avoid a 403 Forbidden error when accessing certain links (e.g., noaa.gov) +user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" + +# Ignore working links that cause a linkcheck 403 error. +linkcheck_ignore = [r'https://www\.intel\.com/content/www/us/en/docs/cpp\-compiler/developer\-guide\-reference/2021\-10/thread\-affinity\-interface\.html', + r'https://www\.intel\.com/content/www/us/en/developer/tools/oneapi/hpc\-toolkit\-download\.html', + #r'https://glossary.ametsoc.org/.*', + ] + +# Ignore anchor tags for SRW App data bucket. Shows Not Found even when they exist. +linkcheck_anchors_ignore = [r"current_srw_release_data/", + r"input_model_data/.*", + r"fix.*", + r"sample_cases/.*", + ] + +linkcheck_allowed_redirects = {r"https://github\.com/ufs-community/ufs-srweather-app/wiki/.*": + r"https://raw\.githubusercontent\.com/wiki/ufs-community/ufs-srweather-app/.*", + r"https://github\.com/ufs-community/ufs-srweather-app/issues/new/choose": + r"https://github\.com/login", + r"https://doi\.org/.*/zenodo\..*": r"https://zenodo\.org/records/.*", + r"https://doi\.org/.*": r"https://gmd\.copernicus\.org/.*", + r"https://rdhpcs\-common\-docs\.rdhpcs\.noaa\.gov/wiki/index\.php/Transferring\_Data": + r"https://sso\.noaa\.gov\:443/openam/SSORedirect/metaAlias/noaa\-online/idp\?SAMLRequest\=.*", + r"https://github\.com/ufs-community/ufs\-srweather\-app/issues/.*": + r"https://github\.com/login\?return\_to\=https.*", + } + + # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for @@ -124,6 +129,7 @@ # html_theme = 'sphinx_rtd_theme' html_theme_path = ["_themes", ] +html_logo= "https://github.com/ufs-community/ufs/wiki/images/ufs-epic-logo.png" # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the @@ -132,7 +138,7 @@ # html_theme_options = {} html_theme_options = { "body_max_width": "none", - "navigation_depth": 6, + "navigation_depth": 8, } # Add any paths that contain custom static files (such as style sheets) here, @@ -191,7 +197,7 @@ def setup(app): # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'UFS-SRWeatherApp.tex', 'UFS Short-Range Weather App Users Guide', + (master_doc, 'UFS-SRWeatherApp.tex', 'UFS Short-Range Weather App Documentation', ' ', 'manual'), ] @@ -201,7 +207,7 @@ def setup(app): # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - (master_doc, 'UFS-SRWeatherApp', 'UFS Short-Range Weather App Users Guide', + (master_doc, 'UFS-SRWeatherApp', 'UFS Short-Range Weather App Documentation', [author], 1) ] @@ -253,16 +259,16 @@ def setup(app): 'ccpp-techdoc': ('https://ccpp-techdoc.readthedocs.io/en/ufs_srw_app_v2.2.0/', None), 'stochphys': ('https://stochastic-physics.readthedocs.io/en/latest/', None), 'srw_v2.2.0': ('https://ufs-srweather-app.readthedocs.io/en/release-public-v2.2.0/', None), - 'uw': ('https://uwtools.readthedocs.io/en/main', None), } # -- Options for extlinks extension --------------------------------------- extlinks_detect_hardcoded_links = True -extlinks = {'srw-wiki': ('https://github.com/ufs-community/ufs-srweather-app/wiki/%s','%s'), - } - -# -- Options for todo extension ---------------------------------------------- - -# If true, `todo` and `todoList` produce output, else they produce nothing. -todo_include_todos = True +extlinks = {'github-docs': ('https://docs.github.com/en/%s', '%s'), + 'nco': ('https://www.nco.ncep.noaa.gov/idsb/implementation_standards/%s', '%s'), + "rst": ("https://www.sphinx-doc.org/en/master/usage/restructuredtext/%s", "%s"), + "rtd": ("https://readthedocs.org/projects/ufs-srweather-app/%s", "%s"), + 'srw-repo': ('https://github.com/ufs-community/ufs-srweather-app/%s', '%s'), + 'srw-wiki': ('https://github.com/ufs-community/ufs-srweather-app/wiki/%s','%s'), + 'uw': ('https://uwtools.readthedocs.io/en/main/%s', '%s'), + } \ No newline at end of file diff --git a/doc/index.rst b/doc/index.rst new file mode 100644 index 0000000000..c8cf2b32fc --- /dev/null +++ b/doc/index.rst @@ -0,0 +1,9 @@ +UFS Short-Range Weather App Documentation (|version|) +===================================================== + +.. toctree:: + :numbered: + :maxdepth: 3 + + UsersGuide/index + ContribGuide/index diff --git a/doc/UsersGuide/make.bat b/doc/make.bat similarity index 100% rename from doc/UsersGuide/make.bat rename to doc/make.bat diff --git a/doc/UsersGuide/source/references.bib b/doc/references.bib similarity index 100% rename from doc/UsersGuide/source/references.bib rename to doc/references.bib diff --git a/doc/UsersGuide/requirements.in b/doc/requirements.in similarity index 100% rename from doc/UsersGuide/requirements.in rename to doc/requirements.in diff --git a/doc/UsersGuide/requirements.txt b/doc/requirements.txt similarity index 97% rename from doc/UsersGuide/requirements.txt rename to doc/requirements.txt index 60c67635ea..0671225d72 100644 --- a/doc/UsersGuide/requirements.txt +++ b/doc/requirements.txt @@ -12,7 +12,7 @@ certifi==2024.2.2 # via requests charset-normalizer==3.3.2 # via requests -docutils==0.19 +docutils==0.20.1 # via # pybtex-docutils # sphinx @@ -58,7 +58,7 @@ sphinx-rtd-theme==2.0.0 # via -r requirements.in sphinxcontrib-applehelp==1.0.8 # via sphinx -sphinxcontrib-bibtex==2.5.0 +sphinxcontrib-bibtex==2.6.2 # via -r requirements.in sphinxcontrib-devhelp==1.0.6 # via sphinx diff --git a/doc/UsersGuide/source/tables/SRW_NATLEV_table.csv b/doc/tables/SRW_NATLEV_table.csv similarity index 100% rename from doc/UsersGuide/source/tables/SRW_NATLEV_table.csv rename to doc/tables/SRW_NATLEV_table.csv diff --git a/doc/UsersGuide/source/tables/SRW_NATLEV_table.rst b/doc/tables/SRW_NATLEV_table.rst similarity index 100% rename from doc/UsersGuide/source/tables/SRW_NATLEV_table.rst rename to doc/tables/SRW_NATLEV_table.rst diff --git a/doc/UsersGuide/source/tables/SRW_PRSLEV_table.csv b/doc/tables/SRW_PRSLEV_table.csv similarity index 100% rename from doc/UsersGuide/source/tables/SRW_PRSLEV_table.csv rename to doc/tables/SRW_PRSLEV_table.csv diff --git a/doc/UsersGuide/source/tables/SRW_PRSLEV_table.rst b/doc/tables/SRW_PRSLEV_table.rst similarity index 100% rename from doc/UsersGuide/source/tables/SRW_PRSLEV_table.rst rename to doc/tables/SRW_PRSLEV_table.rst diff --git a/doc/UsersGuide/source/tables/Tests.csv b/doc/tables/Tests.csv similarity index 100% rename from doc/UsersGuide/source/tables/Tests.csv rename to doc/tables/Tests.csv diff --git a/doc/UsersGuide/source/tables/Tests.rst b/doc/tables/Tests.rst similarity index 100% rename from doc/UsersGuide/source/tables/Tests.rst rename to doc/tables/Tests.rst diff --git a/doc/tables/code-managers.csv b/doc/tables/code-managers.csv new file mode 100644 index 0000000000..ec665b0fa7 --- /dev/null +++ b/doc/tables/code-managers.csv @@ -0,0 +1,21 @@ +Affiliation;Code Manager;Areas of Expertise +EPIC;**Michael Lueken (@MichaelLueken)**;Lead SRW App code manager +GSL;Daniel Abdi (@danielabdi-noaa);Workflow generation, testing RRFS on the cloud, environment modules +GSL;Jeff Beck (@JeffBeck-NOAA);SRW App configuration/workflow, code management, meteorological evaluation +EMC;Ben Blake (@BenjaminBlake-NOAA);Output visualization, Rocoto +EMC;Brian Curtis (@BrianCurtis-NOAA);Operational air quality modeling (Online-CMAQ), code management +GSL;Christopher Harrop (@christopherwharrop-noaa);Rocoto, code management, and testing +GSL;Christina Holt (@christinaholtNOAA);Workflow, conda environment support, testing, and code management +EPIC;Chan-Hoo Jeon (@chan-hoo);Air quality modeling (Online-CMAQ), NCO Implementation Standards, Workflow +EPIC;Jong Kim (@jkbk2004);UFS Weather Model configuration, forecast sensitivity analysis, data assimilation +NCAR;Mike Kavulich (@mkavulich);Workflow, CCPP/physics, code management, WE2E testing, verification +GSL;Gerard Ketefian (@gsketefian);Verification/METplus tasks, Jinja templates, and workflow scripts +NCAR;Will Mayfield (@willmayfield);Verification/METplus tasks +GSL;Linlin Pan (@panll);Workflow, CCPP/physics, and verification +EPIC;Natalie Perlin (@natalie-perlin);Generic Linux/Mac installations, hpc-stack/spack-stack +EPIC;Gillian Petro (@gspetro-NOAA);Documentation, User Support +EPIC;Mark Potts (@mark-a-potts);HPC systems +EPIC;Edward Snyder (@EdwardSnyder-NOAA);WE2E testing, input data +GLERL;David Wright (@dmwright526);FVCOM integration, output visualization, preprocessing tasks +EPIC;Ratko Vasic (@RatkoVasic-NOAA);Workflow, testing, and spack-stack maintenance +NSSL;Yunheng Wang (@ywangwof);HPC systems, code management, and regional workflow (especially on Stampede, Jet and NSSL computers) diff --git a/doc/UsersGuide/source/tables/fix_file_list.rst b/doc/tables/fix_file_list.rst similarity index 100% rename from doc/UsersGuide/source/tables/fix_file_list.rst rename to doc/tables/fix_file_list.rst