Merge pull request #2 from MeteoSwiss-APN/new-plots

 New plots (station, daytime, time, ensemble), Add configuration

.gitignore

@@ -1,5 +1,6 @@
 # remove images
 *.png
+!img/*
 
 # remove scratch files
 # C extensions

README.md

@@ -1,106 +1,142 @@
-# MOVERO PLOTS
-## 0. GENERAL
-### 0.1 Task Description / Overview
-> **_Task_**: Replace the IDL plot scripts for the verification plots with Python scripts.
-> There is a number of different plots, which need to be created. For each type of plot a number of scripts is necessary.
-
-These plots need to be implemented:
-1. Time Series of verification scores
-2. Diurnal cycle of verification scores
-3. Total scores depending on lead-time ranges
-4. Numeric values of total scores
-
-
-
-### 0.2 Usage
-The entry point for this package is a script called [plot_synop](src/movero/plot_synop.py). By executing `python plot_synop.py --help`, one can see the required and optional command line inputs:
+# MOVEROPLOT
+Moveroplot is a tool for the production of various verification graphics including line graphs, maps, histograms, and reliability diagrams
+based on verification results produced by Movero.
+## Getting Started
+To setup the project, follow the instructions in [CONTRIBUTING.md](CONTRIBUTING.md).
+
+
+## Usage
+The primary command for moveroplot follows the structure
+```bash
+moveroplot [OPTIONS] MODEL_VERSION(S)
+moveroplot [OPTIONS] MODEL_VERSION(S)
 ```
-Usage: plot_synop.py [OPTIONS] MODEL_VERSION
-
-  Entry Point for the MOVERO Plotting Pipeline.
-
-  The only input argument is the MODEL_VERSION argument. Pass this along with any number
-  of  options. These usually have a default value or are not necessary.
-
+To see the available options, run:
+```bash
+moveroplot -h
+```
+```
+The following options are listed below:
 Options:
-  --debug                         Add debug comments to command prompt.
-  --lt_ranges TEXT                Specify the lead time range(s) of interest.
-                                  Def: 19-24
-  --plot_params TEXT              Specify parameters to plot.
-  --plot_scores TEXT              Specify scores to plot.
-  --plot_cat_params TEXT          Specify categorical parameters to plot.
-  --plot_cat_thresh TEXT          Specify categorical scores thresholds to
-                                  plot.
-  --plot_cat_scores TEXT          Specify categorical scores to plot.
-  --plot_ens_params TEXT          Specify ens parameters to plot.
-  --plot_ens_thresh TEXT          Specify ens scores thresholds to plot.
-  --plot_ens_scores TEXT          Specify ens scores thresholds to plot.
-  --input_dir PATH                Specify input directory.
-  --output_dir TEXT               Specify output directory. Def: plots
-  --relief                        Add relief to maps.
-  --grid                          Add grid to plots.
-  --season [2020s4|2021s1|2021s2|2021s3|2021s4]
-                                  Specify the season of interest. Def: 2021s4
-  --help                          Show this message and exit.
+  --plot_type TEXT            Specify the type of plot to generate: [total,
+                              time, station, daytime, ensemble].
+  --debug                     Add debug comments to command prompt.
+  --lt_ranges TEXT            Specify the lead time ranges of interest. Def:
+                              19-24
+  --plot_params TEXT          Specify parameters to plot.
+  --plot_scores TEXT          Specify scores to plot.
+  --plot_cat_params TEXT      Specify categorical parameters to plot.
+  --plot_cat_thresh TEXT      Specify categorical scores thresholds to plot.
+  --plot_cat_scores TEXT      Specify categorical scores to plot.
+  --plot_ens_params TEXT      Specify parameters to ensemble plots.
+  --plot_ens_scores TEXT      Specify scores to ensemble plots.
+  --plot_ens_cat_params TEXT  Specify categorical parameters to ensemble
+                              plots.
+  --plot_ens_cat_scores TEXT  Specify categorical scores to ensemble plots.
+  --plot_ens_cat_thresh TEXT  Specify categorical scores thresholds to
+                              ensemble plots.
+  --input_dir PATH            Specify input directory.
+  --output_dir TEXT           Specify output directory. Def: plots
+  --colors TEXT               Specify the plot color for each model version
+                              using matploblib's color coding
+  --relief                    Add relief to maps.
+  --grid                      Add grid to plots.
+  -V, --version               Show the version and exit.
+  -v, --verbose               Increase verbosity; specify multiple times for
+                              more.
+  -h, --help                  Show this message and exit.
 
 ```
+`moveroplot` efficiently processes user inputs to construct a `plot_setup` dictionary, which is pivotal in organizing the plotting process.
+This dictionary is structured with two primary keys: 'model_versions' and 'parameter'.
+* 'model_versions': This key maps to a list encompassing the model versions to plot.
+* 'parameter': This key connects to a nested dictionary. Within this nested structure, each parameter serves as a key linked to its corresponding scores to plot.
 
-> [time=Wed, Mar 2, 2022 2:21 PM]
-**Command so far to create all plots for model v. C-1E-CTR_ch:**
-
+To offer a clearer understanding, the image below illustrates the potential parameters and their associated scores and their thresholds:
+![**Parameters Dictitonary**](https://i.imgur.com/kdQrufu.png)
 
+In the subsequent stages, `plot_setup` is channeled into distinct plotting pipelines. There, the source files are retrieved, parsed and plotted.
+Ultimately, all plots are saved in the `<output_dir>/` directory as PNG files.
 
+### Usage Examples
+Example Command plotting Station, Time, Total and Daytime Scores:
 ```
-python plot_synop.py C-1E-CTR_ch
---plot_params TOT_PREC12,TOT_PREC6,TOT_PREC1,CLCT,GLOB,DURSUN12,DURSUN1,T_2M,T_2M_KAL,TD_2M,TD_2M_KAL,RELHUM_2M,FF_10M,FF_10M_KAL,VMAX_10M6,VMAX_10M1,DD_10M,PS,PMSL
---plot_scores ME,MMOD/MOBS,MAE,STDE,RMSE,COR,NOBS
---plot_cat_params TOT_PREC12,TOT_PREC6,TOT_PREC1,CLCT,T_2M,T_2M_KAL,TD_2M,TD_2M_KAL,FF_10M,FF_10M_KAL,VMAX_10M6,VMAX_10M1
+moveroplot C-1E_ch/C-2E_ch --lt_ranges 07-12,19-24,61-72 --input_dir /scratch/osm/movero/wd/2022s4 --plot_type station,time,daytime,total
+--plot_cat_params TOT_PREC12,TOT_PREC6,CLCT,T_2M,TD_2M,FF_10M,VMAX_10M6
 --plot_cat_thresh 0.1,1,10:0.2,1,5:0.2,0.5,2:2.5,6.5:0,15,25:0,15,25:-5,5,15:-5,5,15:2.5,5,10:2.5,5,10:5,12.5,20:5,12.5,20
 --plot_cat_scores FBI,MF/OF,POD,FAR,THS,ETS
+--plot_params TOT_PREC12,TOT_PREC6,TOT_PREC1,CLCT,GLOB,DURSUN12,DURSUN1,T_2M,T_2M_KAL,TD_2M,TD_2M_KAL,RELHUM_2M,FF_10M,FF_10M_KAL,VMAX_10M6,VMAX_10M1,DD_10M,PS,PMSL
+--plot_scores ME,MMOD/MOBS,MAE,STDE,RMSE,COR,NOBS
+```
+Example Command plotting Ensemble Scores:
+```
+moveroplot C-1E_ch/C-2E_ch --lt_ranges 07-12,19-24,61-72 --input_dir /scratch/osm/movero/wd/2022s4 --plot_type ensemble
+--plot_ens_params TOT_PREC12,TOT_PREC6,CLCT,T_2M,TD_2M,FF_10M,VMAX_10M6
+--plot_ens_scores OUTLIERS,RANK,RPS,RPS_REF
+--plot_ens_cat_params TOT_PREC12,TOT_PREC6,CLCT,T_2M,TD_2M,FF_10M,VMAX_10M6
+--plot_ens_cat_thresh 0.1,0.2,2.5,0,0,2.5,5
+--plot_ens_cat_scores REL,RES,BS,BS_REF,BSS,BSSD,REL_DIA
 ```
 
-`plot_synop.py` parses these user inputs into a _parameter dictionary_. Each provided parameter is one key in this dictionary. For every key, a list of corresponding scores is assigned.
-![**Parameters Dictitonary**](https://i.imgur.com/kdQrufu.png)
-Afterwards this `params_dict` is passed to separate plotting pipelines. There, the source files are retrieved, parsed and plotted. Ultimately, all plots are placed in the `<output_dir>/<plot_type>/` directory.
-
-## 1. SPATIAL VERIFICATION
-<!-- ![alt text](http://i.imgur.com/8o44hib.png) -->
-
-> Relevant File: [station_score.py](src/movero/station_scores.py)
-
-The spatial verification plots feature a map, where all stations have are marked with a coloured dot. The colour of this dot corresponds to a colour-bar on the right side of the map. The smaller the deviation from the centre of the colourbar, the better. One can see directly, if & where the model performed well, or rather less so.
-
-###### Example: Old Station Score Plot
-<img src="https://i.imgur.com/LA2zev4.png)" alt="drawing" width="300"/>
-
-###### Example: New Station Scores Plot
-<img src="https://i.imgur.com/SiHcvpW.jpg" alt="drawing" width="300"/>
-
----
-
+## Plotting Pipeline and Output
+### Plotting Multiple Model Versions
+`moveroplot` offers the option to visualize multiple results from distinct model versions within a single plot or image, depending on the plot type.
+This can be achieved through the use of specific delimiters: a slash (/) signifies combined plotting, while a comma (,) indicates separate plots.
 
-## 2. TIME SERIES OF VERIFICATION SCORES
-> Relevant File: [time_scores.py](src/movero/time_scores.py)
+Example:
+> Input: C-1E_ch/C-2E_ch,C-1E_alps
+>
+> Interpretation: Display results of C-1E_ch and C-2E_ch into one combined plot.
+> Plot the results of C-1E_alps separately.
 
-###### Example: Old vs. New Station Scores Plot
-![](https://i.imgur.com/g9t612p.png)
-![](https://i.imgur.com/mlwMtTY.png)
+### Spatial Verification
+> Relevant File: [station_scores.py](src/moveroplot/station_scores.py)
+>
+> Note: Each station score image is consistent in its LT range.
+> The number of plots per image can vary (model versions along columns, scores along rows).
+>
+> Note: Invalid Atab files are ignored.
 
----
-## 3. DIURNAL CYCLYE OF VERIFICATION SCORES
-> Relevant File: [daytime_scores.py](src/movero/daytime_scores.py)
-###### Example: Old vs. New Station Scores Plot
-![](https://i.imgur.com/FGSW1My.png)
-![](https://i.imgur.com/pSNKEF4.png)
+![**Example Station Scores**](img/station_scores_example.png)
 
+### Time Series of Verification Scores
+> Relevant File: [time_scores.py](src/moveroplot/time_scores.py)
+>
+> Remark: The order specified in --plot_scores and --plot_cat_scores is crucial. Two plots are assigned per page. Each threshold and parameter initialize a new page.
+>
+> Remark: Model versions and scores can be displayed in the same plot using `/` in the input.
+>
+![**Example Time Scores**](img/time_scores_example.png)
 
-___
-## 4. TOTAL SCORES DEP. ON LEAD-TIME RANGES
+### Diurnal Cycle of Verification Scores
+> Relevant File: [daytime_scores.py](src/moveroplot/daytime_scores.py)
+>
+> Remark: The order specified in --plot_scores and --plot_cat_scores is crucial. Two plots are assigned per page. Each threshold and parameter initialize a new page.
+>
+> Remark: Model versions and scores can be displayed in the same plot using `/` in the input
+![**Example Daytime Scores**](img/daytime_scores_example.png)
 
-> Remark: how are scores assigned to subplots?
+### Total scores for all lead times
+> Relevant File: [total_scores.py](src/moveroplot/total_scores.py)
+>
+> Remark: The order specified in --plot_scores and --plot_cat_scores is crucial. Four plots are assigned per page. Each threshold and parameter initialize a new page.
 >
-> die Reihenfolge in --plot_scores ist entscheidend. Es kommen immer 4 plots auf eine Seite für die normalen Scores. Die --plot_cat_scores beginnen auf jeden Fall auf einer neuen Seite, und jeder Threshold beginnt wieder auf einer neuen Seite.
+> Remark: Model versions and scores can be displayed in the same plot using `/` in the input
+>
+![**Example Total Scores**](img/total_scores_example.png)
 
-###### Example: Old vs. New Station Scores Plot
-![](https://i.imgur.com/RViAUU4.png)
-![](https://i.imgur.com/2d69BoT.png)
+### Ensemble scores
+> Relevant File: [ensemble_scores.py](src/moveroplot/ensemble_scores.py)
+>
+> Remark: The order specified in --plot_ens_scores and --plot_ens_cat_scores is crucial. Each threshold and parameter initialize a new page.
+>
+> Remark: Model versions and scores can be displayed in the same plot using `/` in the input.
+>
+> Remark: RANK, REL_DIA and line plots are saved in separate images.
+
+#### Regular Line Plots
+![**Example Regular Ensemble Scores**](img/ensemble_scores_OUTLIERS_example.png)
+#### RANK
+![**Example RANK**](img/ensemble_scores_RANK_example.png)
+#### Reliability Diagram
+![**Example REAL DIA**](img/ensemble_scores_REL_DIA_example.png)

src/moveroplot/cli.py

@@ -22,17 +22,22 @@
 )
 @click.version_option(__version__, "--version", "-V", message="%(version)s")
 @click.command(context_settings={"help_option_names": ["-h", "--help"]})
+@click.option(
+    "--plot_type",
+    type=str,
+    help="""Specify the type of plot to generate:
+    [total, time, station, daytime, ensemble].""",
+)
 @click.argument(
-    "model_version", type=str
+    "model_versions", type=str, default="C-1E-CTR_ch,C-1E_ch"
 )  # help="Specify the correct run. I.e. C-1E-CTR_ch"
 @click.option(
     "--debug", type=bool, is_flag=True, help="Add debug comments to command prompt."
 )
 @click.option(
     "--lt_ranges",
     type=str,
-    multiple=True,
-    default=("19-24",),
+    default="19-24",
     help="Specify the lead time ranges of interest. Def: 19-24",
 )
 @click.option("--plot_params", type=str, help="Specify parameters to plot.")
@@ -51,22 +56,33 @@
 # 0.1,1,10:0.2,1,5:0.2,0.5,2:2.5,6.5:0,15,25:0,15,25:-5,5,
 # 15:-5,5,15:2.5,5,10:2.5,5,10:5,12.5,20:5,12.5,20
 @click.option("--plot_cat_scores", type=str, help="Specify categorical scores to plot.")
-# FBI,MF,POD,FAR,THS,ETS
 @click.option(
-    "--plot_ens_params", type=str, help="Specify ens parameters to plot."
-)  # TODO: figure out what ens params are
+    "--plot_ens_params", type=str, help="Specify parameters to ensemble plots."
+)
+@click.option("--plot_ens_scores", type=str, help="Specify scores to ensemble plots.")
 @click.option(
-    "--plot_ens_thresh", type=str, help="Specify ens scores thresholds to plot."
-)  # TODO: figure out what ens thresh are
+    "--plot_ens_cat_params",
+    type=str,
+    help="Specify categorical parameters to ensemble plots.",
+)
+@click.option(
+    "--plot_ens_cat_scores",
+    type=str,
+    help="Specify categorical scores to ensemble plots.",
+)
 @click.option(
-    "--plot_ens_scores", type=str, help="Specify ens scores thresholds to plot."
-)  # TODO: figure out what ens scores are
+    "--plot_ens_cat_thresh",
+    type=str,
+    help="Specify categorical scores thresholds to ensemble plots.",
+)
+# FBI,MF,POD,FAR,THS,ETS
 # C-1E-CTR_ch
 # 🔰 new options for plot_synop call
 @click.option(
     "--input_dir",
     type=click.Path(exists=True),
-    default=Path("/scratch/osm/movero/wd"),
+    default=Path("/scratch/osm/movero/wd/2022s4"),
+    # default=Path("/scratch/kaufmann/movero/wd/2023s3_icon"),
     help="Specify input directory.",
 )
 @click.option(
@@ -75,23 +91,14 @@
     default=Path("plots"),
     help="Specify output directory. Def: plots",
 )
-@click.option("--relief", type=bool, is_flag=True, help="Add relief to maps.")
-@click.option("--grid", type=bool, is_flag=True, help="Add grid to plots.")
 @click.option(
-    "--season",
-    type=click.Choice(
-        [
-            "2020s4",
-            "2021s1",
-            "2021s2",
-            "2021s3",
-            "2021s4",
-        ]
-    ),
-    multiple=False,
-    default="2021s4",
-    help="Specify the season of interest. Def: 2021s4",
+    "--colors",
+    type=str,
+    help="""Specify the plot color for each model version
+    using matploblib's color coding""",
 )
+@click.option("--relief", type=bool, is_flag=True, help="Add relief to maps.")
+@click.option("--grid", type=bool, is_flag=True, help="Add grid to plots.")
 @click.pass_context
 def cli(ctx: Context, **kwargs) -> None:
     """Console script for test_cli_project."""

src/moveroplot/config/plot_settings.py

@@ -0,0 +1,16 @@
+"""Static configurations settings for plots."""
+
+modelcolors: list[str] = [
+    "black",
+    "red",
+    "blue",
+    "green",
+    "cyan",
+    "yellow",
+    "magenta",
+    "orange",
+]
+
+line_styles: list[str] = ["-", ":", "--", "-."]
+
+marker_styles: list[str] = ["D", "^", "o", "v"]