Merge pull request #185 from chhoumann/background-ensemble-learning-r…

…ewrite

Background rewrite ensemble learning

report_thesis/src/sections/background/ensemble_learning_models/ensemble_learning.tex

@@ -1,8 +1,7 @@
 \subsubsection{Ensemble Learning}
-Ensemble learning is a technique in machine learning where multiple models, known as \textit{weak learners}, are combined to produce more accurate predictions.
-Mathematically, ensemble learning can be defined as combining the predictions of $M$ weak learners to form a final prediction $\hat{y}$, such that:
-\begin{equation}
-    \hat{y} = \sum_{m=1}^{M} \alpha_m \hat{y}_m,
-\end{equation}
-where $\hat{y}_m$ is the prediction of the $m$-th weak learner and $\alpha_m$ is the weight assigned to the $m$-th weak learner.
-While there are various choices for weak learners, decision trees are a common choice\cite{James2023AnIS}.
+Ensemble learning is a machine learning technique that combines multiple models, referred to as weak learners, to generate more accurate predictions.
+Individually, the predictive ability of weak learners may be limited, but when combined, they can produce a more precise and robust model.
+Ensemble learning encompasses several methods, including bagging, boosting, and stacking\cite{James2023AnIS, pavlyshenko2018stacking}
+
+In this section, we provide an overview of the ensemble learning methods used in this work.
+We begin by introducing decision trees, a commonly used weak learner in ensemble methods.

report_thesis/src/sections/background/ensemble_learning_models/index.tex

@@ -7,4 +7,5 @@ \subsection{Ensemble Learning Models}
 \input{sections/background/ensemble_learning_models/etr.tex}
 \input{sections/background/ensemble_learning_models/gbr.tex}
 \input{sections/background/ensemble_learning_models/ngboost.tex}
-\input{sections/background/ensemble_learning_models/xgboost.tex}
+\input{sections/background/ensemble_learning_models/xgboost.tex}
+\input{sections/background/ensemble_learning_models/stacked_generalization.tex}

report_thesis/src/sections/background/stacked_generalization.tex → report_thesis/src/sections/background/ensemble_learning_models/stacked_generalization.tex

@@ -1,9 +1,7 @@
-\subsection{Stacked Generalization}
-Stacked generalization, introduced by \citet{wolpertstacked_1992}, is a method designed to improve the predictive performance of machine learning models by leveraging the strengths of multiple models.
-
-In this technique, multiple base models are trained on the original dataset.
-The outputs of these base models serve as inputs to a meta-model, which is trained to make the final prediction.
-This strategy enables the meta-model to learn the optimal way to combine the outputs of the base models to minimize the generalization error.
+\subsubsection{Stacked Generalization}
+Stacked generalization, introduced by \citet{wolpertstacked_1992}, is an ensemble method that combines the predictions of multiple base models, which are trained on the original dataset, as input to a meta-model.
+This meta-model is trained to make the final prediction.
+The strategy allows the meta-model to learn the optimal way to combine the predictions of the base models to minimize the generalization error.
 
 Formally, let $\mathbf{X}$ denote the input data and $\mathbf{y}$ the target variable.
 Initially, $N$ base models $G_1, G_2, \ldots, G_N$ are trained on the dataset $\mathbf{X}$.

report_thesis/src/sections/background/index.tex

@@ -5,5 +5,4 @@ \section{Background}\label{sec:background}
 
 \input{sections/background/preprocessing/index.tex}
 \input{sections/background/ensemble_learning_models/index.tex}
-\input{sections/background/linear_and_regularization_models/index.tex}
-\input{sections/background/stacked_generalization.tex}
+\input{sections/background/linear_and_regularization_models/index.tex}