Merge pull request #129 from NOAA-GFDL/add.fre.pp.README

Add.fre.pp.readme

README.md

@@ -9,7 +9,7 @@ FMS Runtime Environment (FRE) CLI developed using Python's Click package
 
 ## **Background**
 
-As part of fre/canopy, MSD wanted to develop a modern, user-friendly CLI that will allow users to call upon FRE commands using a **_fre_** **tool** _subtool_ syntax. Developed with Click, a Python package easily installable through PyPI and Conda, the main goal of this is to allow users access to most, if not all of MSD-managed tools and workflows from one packaged, centralized CLI.
+`fre-cli` is a modern, user-friendly CLI that will allow users to call FRE commands using a **_fre_** **tool** _subtool_ syntax. Leveraging Click, an easily installable Python package available via PyPI and/or Conda, `fre-cli` gives users intuitive and easy-to-understand access to many FRE tools and workflows from one packaged, centralized CLI.
 
 ![Screenshot from 2024-04-18 13-42-04](https://github.com/NOAA-GFDL/fre-cli/assets/98476720/43c028a6-4e6a-42fe-8bec-008b6758ea9b)
 
@@ -57,12 +57,14 @@ Following the instructions above, the user will be able to run `fre` from any di
 ## **Checklist: Currently Implemented Tools**
 
 To be developed:
-- [ ]  **fre check**
+
 - [x]  **fre app**
 - [x]  **fre catalog**
+- [ ]  **fre check**
+- [x]  **fre cmor**
 - [ ]  **fre list**
 - [x]  **fre make**
-- [x]  **fre cmor**
+- [x]  **fre pp**
 - [ ]  **fre run**
 - [ ]  **fre test**
 - [ ]  **fre yamltools**

docs/subtools.rst

@@ -1,20 +1,32 @@
 Subtools
 ========
 
-frecheck
+fre app
 --------
 
-frelist
--------
+fre catalog
+--------
 
-fremake
--------
+fre check (not yet implemented)
+--------
 
-frepp
------
+fre cmor
+--------
 
-frerun
-------
+fre list (not yet implemented)
+--------
 
-fretest
--------
+fre make
+--------
+
+fre pp
+--------
+
+fre run (not yet implemented)
+--------
+
+fre test (not yet implemented)
+--------
+
+fre yamltools (not yet implemented)
+--------

fre/pp/README.md

@@ -1,3 +1,9 @@
+
+
+
+
+
+<!-- this section seems more general than should be in fre/pp
 # **Frepp Canopy**
 _Brief description of tool group's purpose._
 
@@ -15,7 +21,6 @@ The [tool group] fre-cli tools are described below ([Subcommands](#subcommands))
 * Refer to fre-cli [README.md](https://github.com/NOAA-GFDL/fre-cli/blob/main/README.md) for foundational fre-cli usage guide and tips.
 * [tool group] package repository located at: _insert permalink_
 
-
 ## Subcommands
 - `fre [tool group] [subcommand] [options]`
    - Purpose: _Insert subcommand purpose_
@@ -35,6 +40,100 @@ fre [tool group] [subcommand] -[short/long command 1] [argument(s) 1] -[short/lo
 
 etc.
 ```
+-->
+
+## **Quickstart instructions to postprocess FMS history output on PP/AN or gaea**
+
+1. Checkout postprocessing workflow template
+This will clone the postprocessing repository into `/home/$USER/cylc-src/EXPNAME__PLATFORM__TARGET`.
+```
+module load fre/canopy
+fre pp checkout -e EXPNAME -p PLATFORM -t TARGET
+```
+
+2. Configure pp template with either XML or pp.yaml
+
+```
+fre pp configure-xml -e EXPNAME -p PLATFORM -t TARGET -x XML
+```
+or
+```
+fre pp configure-yaml -e EXPNAME -p PLATFORM -t TARGET -y YAML 
+
+```
+
+3. (OPTIONAL BUT RECCOMENDED) Create `history-manifest` for config validation
+
+Create a `history-manifest` of a single tar file archive first for use in the validation. 
+This list represents the available source files within the history tar archives, and enables the 
+validation procedure to catch a wider variety of potential errors. This can be done like so-
+```
+tar -tf /archive/$USER/path/to/history/files/YYYYMMDD.nc.tar | grep -v tile[2-6] | sort > /home/$USER/cylc-src/EXPNAME__PLATFORM__TARGET/history-manifest
+```
+
+4. Validate the configuration
+```
+fre pp validate -e EXPNAME -p PLATFORM -t TARGET
+```
+
+Warnings related to directories are probably valid and should be fixed in `rose-suite.conf`, or created as necessary via `mkdir`.
+
+If you are running postprocessing gaea, you'll need to change the `SITE` variable in `rose-suite.conf` from `ppan` to `gaea`.
+
+5. Install the workflow
+
+```
+fre pp install -e EXPNAME -p PLATFORM -t TARGET
+```
+
+If you are attempting this on gaea, you'll need to make two one-time changes before installing.
+- Currently, `cylc`, `rose`, and `isodatetime` must be in your PATH for new shells. One approach to do this is
+to symlink the fms-user-installed fre-cli cylc/rose/isodatetime scripts into your local `~/bin` directory,
+and then add that `~/bin` directory to your PATH in your `.bashrc` or `.cshrc`. (If you don't do this, Cylc tasks
+will fail complaining those 3 tools are not available.)
+
+```
+cd ~/bin
+ln -s /ncrc/home2/Flexible.Modeling.System/conda/envs/fre-cli/bin/{cylc,rose,isodatetime} .
+echo 'setenv PATH ${PATH}:~/bin' >> ~/.cshrc
+```
+- Currently, the cylc available on gaea (through `module load cylc` or the `PATH` trick above) does not
+include any global configuration, so you'll need to create a file `~/.cylc/flow/global.cylc` that contains the following.
+If you don't do this, Cylc will use your home directory for the scratch space and rapidly fill your quota.)
+
+```
+[install]
+    [[symlink dirs]]
+        [[[localhost]]]
+            run = /gpfs/f5/scratch/gfdl_f/$USER
+```
+
+6. Run the workflow
+
+```
+fre pp run -e EXPNAME -p PLATFORM -t TARGET
+```
+
+7. Report status of workflow progress
+
+```
+fre pp status -e EXPNAME -p PLATFORM -t TARGET
+```
+
+8. Launch GUI
+
+```
+TODO: fre pp gui?
+
+The full GUI can be launched on jhan or jhanbigmem (an107 or an201).
+
+cylc gui --ip=`hostname -f` --port=`jhp 1` --no-browser
+```
+
+## **PP Wrapper Usage**
+
+Here
+
 ### **PP Wrapper Decision Tree**
 ![pp_wrapper_decsiontree](https://github.com/NOAA-GFDL/fre-cli/assets/98476720/d3eaa237-1e29-4922-9d83-8d9d11925c54)