Merge pull request #2 from SZtools/dev

Dev v 2.0.0

.gitignore

@@ -1,2 +1,4 @@
 __pycache__/
-scripts/__pycache__/
+scripts/__pycache__/
+installer/__pycache__/
+images/__pycache__

README.md

@@ -1,50 +1,57 @@
 # Susceptibility Zoning plugin (SZ)
 
 
+The latest plugin docs are available to: https://sz-docs.readthedocs.io
+
+Plugin repository: https://github.com/SZtools/SZ-plugin
+
+
 ## Introduction
 
-This repository contains the code for a plugin for [QGIS](https://www.qgis.org), called "Susceptibility Zoning plugin" (SZ-plugin), aiming at creating maps of susceptibility to various natural forcing elements.
+This repository contains the code for a [QGIS](https://www.qgis.org) plugin, called "Susceptibility Zoning plugin" (SZ-plugin), aiming at creating maps of susceptibility to various natural forcing elements.
 
 The plugin has been developed with a main focus and application towards landslides susceptibility, but it can be applied to different types of natural events or inter-disciplinary applications.
 
-The plugin uses several type of statistical model for susceptibility evaluation, such as:
+The plugin uses several type of statistical, machine learning models for susceptibility evaluation, such as:
 
-* Weight of Evidence
-* Frequency Ratio
-* Logistic Regression
-* Decision Tree
-* Support Vector Machine
-* Random Forest
+* Generalized Additive Model with two distributions: Binomial, Gaussian (see [pygam](https://pygam.readthedocs.io))
+* Decision Tree (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html))
+* Support Vector Machine (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html))
+* Random Forest (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier))
+* Multi-layer Perceptron as classifier (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html#sklearn.neural_network.MLPClassifier)) and regressor (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html#sklearn.neural_network.MLPRegressor))
 
-The plugin allows to cross-validate the results by simple random selection of test/train samples or allows to cross-validate by k-fold cross-validation method.
+The plugin allows the user to perform spatial and temporal analysis implementing space-time cross-validation (CV) methods: 
 
+* Kfold CV (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html))
+* Spatial CV (see [Elia et al., 2023](https://doi.org/10.1016/j.scitotenv.2023.165289))
+* Time Series Split temporal-CV (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html))
+* Leave One Out temporal-CV (see [scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.LeaveOneOut.html))
+* Space-time CV (Leave One Out/Spatial CV)
+
+Moreover the plugin allows the user to run space-time transferability with all the models implemented.
 
 ## Installation
 
 The SZ plugin is not an official QGIS plugin.
 
 #### Minimum requirements
 
-Minimum tested version of QGIS is 3.28
-
 Tested on:
 
 * Ubuntu 20.04 - QGIS 3.34
+* Ubuntu 22.04 - QGIS 3.34
 * Windows 10 - QGIS 3.28/QGIS 3.34
 * MacOS Sonoma 14.4 - QGIS 3.28
 
 #### Download and install
 
 It can be installed on QGIS >= 3.28 adding the Plugin Repository to QGIS ```manage and install plugins```:
-- flag ```show also experimental plugins```
-
-<img src="./images/experimental.png" width="500">
 
 - add plugin repository
 
 <img src="./images/addrepo.png" width="500">
 
-- edit repository and add [https://sztools.github.io/SZ/sz.xml](https://sztools.github.io/SZ/sz.xml) (THIS REPOSITORY IS TEMPORARY)
+- edit repository and add [https://sztools.github.io/SZ/sz.xml](https://sztools.github.io/SZ/sz.xml)
 
 <img src="./images/repodetail.png" width="500">
 <p>
@@ -72,22 +79,68 @@ At the end you should have the SZ plugin in your processing toolbox
 
 ### GUI
 
-The functions are grouped into 3 cathegories:
+The functions are grouped into 4 cathegories:
+
 * _Data preparation_
-* _SI_
-* _SI k-fold_
+* _Modelling_
+* _SI transfer_
 * _Classify SI_
 
-_Data preparation_ functions can be used for data pre-processing
-_SI_ functions run the statistic models for susceptibility, cross-validate by a simple random selection of train/test samples and evaluate the prediction capacity by ROC curves
-_SI k-fold_ functions run the statistic models for susceptibility, cross-validate by k-fold method and evaluate the prediction capacity by ROC curves
-_Classify SI_ functions allows to cathegorize the susceptibility index into _n_ classes on the base of AUC maximization.
-
-### Input data of SI and SI k-fold functions
+* _Data preparation_ functions can be used for data pre-processing
+* _Modelling_ functions run the statistic models (_Statistical tools_) and machine learning models (_Machine Learning tools_ / _Neural Network tools_) for susceptibility, cross-validate and evaluate the results by several metrics: ROC (AUC), F1 score, Choen's kappa index, Root Meet Square Error, Mean Absolute Error, Squared R, Pearson Correlation
+* _SI transfer_ functions run the statistic models (_Statistical tools_) and machine learning models (_Machine Learning tools_ / _Neural Network tools_) for susceptibility, and apply transfer learning in time and space.
+* _Classify SI_ functions allows the user to cathegorize the susceptibility index, to run ROC analysis and to perform Confusion Matrix.
 
-Input data for SI k-fold or SI functions should be a vector layer with a number of fields for independet variables ans a field for the dependent variable classified binomially: 0 for absence, >0 for presence.
+All the functions run vector analysis
 
 <img src="./images/use.png" width="500">
+
+### Usage
+
+#### Modelling - Machine Learning tools
+
+<img src="./images/ML.png" width="500">
+
+* 1 select the vecor layer to be used as parent layer in the variable selection
+* 2 select the independent variabled from the fields of the input layer. Possible format: linear, ordinal (spline fitted), tensors, cathegorical (integer and continuous from 1 to n)
+* 3 select the dependent variable. In case of bionomial selection the variable is 0/1, otherwise an ordinal n_1...n_m
+* 4 select the algorithm: SVC, DT, RF
+* 5 select one of the possible CV method
+* 6 in case of TSS or LOO or space-time CV selection the Time field is necessary 
+* 7 K value selection in case of random, spatial and space-time CV. If k is equal to 1 then a fitting analysis will be conducted
+* 8 set the vector file destination
+* 9 set the folder destination for the weights, partial effects, plots and metrics outputs 
+
+#### Modelling - Statistical Tools
+
+<img src="./images/statistical.png" width="500">
+
+* 1 select the vecor layer to be used as parent layer in the variable selection
+* 2 select the independent variabled from the fields of the input layer. Possible format: linear, ordinal (spline fitted), tensors, cathegorical (integer and continuous from 1 to n)
+* 3 select the distribution: gaussian or binomial
+* 4 if the distribution selected is binomial, all the variables are scaled with a standard scaler. The scale selection is necessary only for gaussian distribution: linear (standard scaler) or logaritmic
+* 5 select the dependent variable. In case of bionomial selection the variable is 0/1, otherwise an ordinal n_1...n_m
+* 6 select one of the possible CV method
+* 7 in case of TSS or LOO or space-time CV selection the Time field is necessary 
+* 8 K value selection in case of random, spatial and space-time CV. If k is equal to 1 then a fitting analysis will be conducted
+* 9 set the vector file destination
+* 10 set the folder destination for the weights, partial effects, plots and metrics outputs* 
+
+#### Modelling - Neural Network Tools
+
+<img src="./images/NN.png" width="500">
+
+* 1 select the vecor layer to be used as parent layer in the variable selection
+* 2 select the independent variabled from the fields of the input layer. Possible format: linear, ordinal (spline fitted), tensors, cathegorical (integer and continuous from 1 to n)
+* 3 select the dependent variable. In case of bionomial selection the variable is 0/1, otherwise an ordinal n_1...n_m
+* 4 select the algorithm: MLP classifier or MLP regressor
+* 5 if the distribution selected is MLP classifier, all the variables are scaled with a standard scaler. The scale selection is necessary only for MLP regressor: linear ([standard scaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html)) or logaritmic
+* 6 select one of the possible CV method
+* 7 in case of TSS or LOO or space-time CV selection the Time field is necessary 
+* 8 K value selection in case of random, spatial and space-time CV. If k is equal to 1 then a fitting analysis will be conducted
+* 9 set the vector file destination
+* 10 set the folder destination for the weights, partial effects, plots and metrics outputs 
+
 
 ### Test
 
@@ -100,13 +153,17 @@ A dataset and QGIS project are available in [test](./test_data) folder to test t
 
 ## Third-part libraries and plugins used
 
-* [Scikit-learn](https://scikit-learn.org/stable/index.html)
-* [libpysal](https://pysal.org/libpysal/)
+* [scikit-learn](https://scikit-learn.org/stable/index.html)
+* pygam
+* pandas
+* shapely
+* fiona
+* seaborn
 
 
 ## Tutorial
 
-video tutorial: https://www.youtube.com/watch?v=XpsiCkVF11s
+video tutorial SZ v1.x: https://www.youtube.com/watch?v=XpsiCkVF11s
 
 ## Application
 
@@ -134,9 +191,13 @@ Titti, Giacomo, Sarretta, Alessandro, Crema, Stefano, Pasuto, Alessandro, & Borg
 
 ## Credits
 
-Giacomo Titti, Alessandro Sarretta and Luigi Lombardo, Padova, November 2021
+v1.x: Giacomo Titti, Alessandro Sarretta and Luigi Lombardo, Padova, November 2021
+
+please cite: Titti G, Sarretta A, Lombardo L, Crema S, Pasuto A and Borgatti L (2022) Mapping Susceptibility With Open-Source Tools: A New Plugin for QGIS. Front. Earth Sci. 10:842425. doi: 10.3389/feart.2022.842425
+
+v2.x: Giacomo Titti, Luigi Lombardo, Liwei Hu, Pietro Festi, Letizia Elia, Bologna, December 2024
 
-please cite as: Giacomo Titti, Alessandro Sarretta and Luigi Lombardo. (2021). CNR-IRPI-Padova/SZ: SZ plugin (Version v1.0). Zenodo.
+please cite: xxx
 
 ## Contacts
 If you have any problem, please write to [email protected] or create new issue [here](https://github.com/SZtools/SZ-plugin/issues)

sz_module/.env

@@ -0,0 +1,4 @@
+# .env
+PROCESSING_PROVIDER=False
+DEBUG=False
+PLUGIN_NAME=SZ