diff --git a/report_thesis/src/sections/background/ensemble_learning_models/ensemble_learning.tex b/report_thesis/src/sections/background/ensemble_learning_models/ensemble_learning.tex index d9432409..0f2cb311 100644 --- a/report_thesis/src/sections/background/ensemble_learning_models/ensemble_learning.tex +++ b/report_thesis/src/sections/background/ensemble_learning_models/ensemble_learning.tex @@ -1,8 +1,7 @@ \subsubsection{Ensemble Learning} -Ensemble learning is a technique in machine learning where multiple models, known as \textit{weak learners}, are combined to produce more accurate predictions. -Mathematically, ensemble learning can be defined as combining the predictions of $M$ weak learners to form a final prediction $\hat{y}$, such that: -\begin{equation} - \hat{y} = \sum_{m=1}^{M} \alpha_m \hat{y}_m, -\end{equation} -where $\hat{y}_m$ is the prediction of the $m$-th weak learner and $\alpha_m$ is the weight assigned to the $m$-th weak learner. -While there are various choices for weak learners, decision trees are a common choice\cite{James2023AnIS}. \ No newline at end of file +Ensemble learning is a machine learning technique that combines multiple models, referred to as weak learners, to generate more accurate predictions. +Individually, the predictive ability of weak learners may be limited, but when combined, they can produce a more precise and robust model. +Ensemble learning encompasses several methods, including bagging, boosting, and stacking\cite{James2023AnIS, pavlyshenko2018stacking} + +In this section, we provide an overview of the ensemble learning methods used in this work. +We begin by introducing decision trees, a commonly used weak learner in ensemble methods. diff --git a/report_thesis/src/sections/background/ensemble_learning_models/index.tex b/report_thesis/src/sections/background/ensemble_learning_models/index.tex index 9dbfa5ee..310b4135 100644 --- a/report_thesis/src/sections/background/ensemble_learning_models/index.tex +++ b/report_thesis/src/sections/background/ensemble_learning_models/index.tex @@ -7,4 +7,5 @@ \subsection{Ensemble Learning Models} \input{sections/background/ensemble_learning_models/etr.tex} \input{sections/background/ensemble_learning_models/gbr.tex} \input{sections/background/ensemble_learning_models/ngboost.tex} -\input{sections/background/ensemble_learning_models/xgboost.tex} \ No newline at end of file +\input{sections/background/ensemble_learning_models/xgboost.tex} +\input{sections/background/ensemble_learning_models/stacked_generalization.tex} \ No newline at end of file diff --git a/report_thesis/src/sections/background/stacked_generalization.tex b/report_thesis/src/sections/background/ensemble_learning_models/stacked_generalization.tex similarity index 69% rename from report_thesis/src/sections/background/stacked_generalization.tex rename to report_thesis/src/sections/background/ensemble_learning_models/stacked_generalization.tex index f17cc2a4..2dd74108 100644 --- a/report_thesis/src/sections/background/stacked_generalization.tex +++ b/report_thesis/src/sections/background/ensemble_learning_models/stacked_generalization.tex @@ -1,9 +1,7 @@ -\subsection{Stacked Generalization} -Stacked generalization, introduced by \citet{wolpertstacked_1992}, is a method designed to improve the predictive performance of machine learning models by leveraging the strengths of multiple models. - -In this technique, multiple base models are trained on the original dataset. -The outputs of these base models serve as inputs to a meta-model, which is trained to make the final prediction. -This strategy enables the meta-model to learn the optimal way to combine the outputs of the base models to minimize the generalization error. +\subsubsection{Stacked Generalization} +Stacked generalization, introduced by \citet{wolpertstacked_1992}, is an ensemble method that combines the predictions of multiple base models, which are trained on the original dataset, as input to a meta-model. +This meta-model is trained to make the final prediction. +The strategy allows the meta-model to learn the optimal way to combine the predictions of the base models to minimize the generalization error. Formally, let $\mathbf{X}$ denote the input data and $\mathbf{y}$ the target variable. Initially, $N$ base models $G_1, G_2, \ldots, G_N$ are trained on the dataset $\mathbf{X}$. diff --git a/report_thesis/src/sections/background/index.tex b/report_thesis/src/sections/background/index.tex index 80883dce..b85edbe6 100644 --- a/report_thesis/src/sections/background/index.tex +++ b/report_thesis/src/sections/background/index.tex @@ -5,5 +5,4 @@ \section{Background}\label{sec:background} \input{sections/background/preprocessing/index.tex} \input{sections/background/ensemble_learning_models/index.tex} -\input{sections/background/linear_and_regularization_models/index.tex} -\input{sections/background/stacked_generalization.tex} \ No newline at end of file +\input{sections/background/linear_and_regularization_models/index.tex} \ No newline at end of file