diff --git a/doc/Project_Report.pdf b/doc/Project_Report.pdf index 516e88c..1b40281 100644 Binary files a/doc/Project_Report.pdf and b/doc/Project_Report.pdf differ diff --git a/doc/Project_Report.tex b/doc/Project_Report.tex index 0d94bf2..964465e 100644 --- a/doc/Project_Report.tex +++ b/doc/Project_Report.tex @@ -384,8 +384,8 @@ \subsection{Base model training} forced to be within the lungs and the random point was used as the center of the patch. During inference, the k should be large enough to ensure that the lung pixels are covered multiple times. Each patch is then fed into a network to -produce a prediction. The confidence score was calculated for each category by -calculating the percentage of predictions for each class based on the k patches. +produce a prediction. The probability score was calculated for each category by +calculating the mean of all probabilities for each category based on the k patches. The optimization algorithm used during training was the Adam optimizer with a learning rate of 0.00001. An early stopping strategy based on validation performance was applied. The best model is selected among 100 epochs training. @@ -539,7 +539,7 @@ \subsection{Implementation details} on a Windows 10 desktop. Rest of the models are trained on AWS p3.2xlarge EC2 (virtual machine) instances featuring NVIDIA Tesla V100 GPUs on Ubuntu 18.04. The base models are fine tuned using weights from pre-trained -models. The data are augmented with random flips, crops and scaling during the +models. The data is augmented with random flips, crops and scaling during the fine tuning process. We created and trained patch-based versions in the same environment as the @@ -577,10 +577,24 @@ \subsubsection{Segmentation Training} \label{fig:segtrain} \end{figure} +\begin{figure*}[h] + \centering + \includegraphics[width=0.8\textwidth]{../doc/images/original_vs_patched_flannel_f1.png} + \caption{COVID-19 F1 score comparison} + \label{fig:f1score} +\end{figure*} + +\begin{figure*}[!htpb] + \centering + \includegraphics[width=0.8\textwidth]{../doc/images/patched_flannel_covid_19_plot_curve.png} + \caption{Patched FLANNEL PR and ROC Curve} + \label{fig:pf_roccurve} +\end{figure*} + \subsubsection{FLANNEL} In this section, we compare original FLANNEL base models and ensemble -performance with the patched FLANNEL. Because COVID-19 class is heavily +performance with the patched FLANNEL. Since COVID-19 class is heavily imbalanced compared to other categories, overall accuracy would not be the appropriate measure for performance evaluation. It would not be able to show performance increase in COVID-19 detection, So instead we use F1-score for @@ -593,20 +607,19 @@ \subsubsection{FLANNEL} notice significant improvement in all patched base models compared to the original base models for COVID-19 detection. -\begin{figure*}[h] +\begin{figure}[h] \centering - \includegraphics[width=0.8\textwidth]{../doc/images/original_vs_patched_flannel_f1.png} - \caption{COVID-19 F1 score comparison} - \label{fig:f1score} -\end{figure*} + \includegraphics[width=7cm]{../doc/images/ensemble_comparison_roc_curve.png} + \caption{EnsembLe Comparison ROC Curve} + \label{fig:ec_roccurve} +\end{figure} -\begin{figure*}[!htpb] +\begin{figure} \centering - \includegraphics[width=0.8\textwidth]{../doc/images/patched_flannel_covid_19_plot_curve.png} - \caption{Patched FLANNEL PR and ROC Curve} - \label{fig:pf_roccurve} -\end{figure*} - + \includegraphics[width=7cm]{../doc/images/ensemble_comparison_precision_recall_curve.png} + \caption{Ensemble Comparison PR Curve} + \label{fig:ec_prcurve} +\end{figure} In Table \ref{table:resultstats}, we show F1-score for each classification and macro F1-score for all classes. In Table \ref{table:resultstats}, we can see @@ -631,20 +644,6 @@ \subsubsection{FLANNEL} to all predictions irrespective of classes. This shows patched ensemble model performs slightly better in overall predictions. -\begin{figure}[h] - \centering - \includegraphics[width=7cm]{../doc/images/ensemble_comparison_roc_curve.png} - \caption{EnsembLe Comparison ROC Curve} - \label{fig:ec_roccurve} -\end{figure} - -\begin{figure}[h] - \centering - \includegraphics[width=7cm]{../doc/images/ensemble_comparison_precision_recall_curve.png} - \caption{Ensemble Comparison PR Curve} - \label{fig:ec_prcurve} -\end{figure} - We also compare the patched and original FLANNEL performance via confusion matrix as shown in Figure \ref{fig:f_cf} and Figure \ref{fig:p_cf}. Due to @@ -655,7 +654,7 @@ \subsubsection{FLANNEL} models struggle with Pneumonia viral images classification and misclassifies them as Pneumonia bacteria images. -\begin{figure}[h] +\begin{figure} \centering \includegraphics[width=7cm]{../doc/images/base_flannel_cf.png} \caption{Original FLANNEL Confusion Matrix} @@ -663,7 +662,7 @@ \subsubsection{FLANNEL} \end{figure} -\begin{figure}[h] +\begin{figure} \centering \includegraphics[width=7cm]{../doc/images/patched_flannel_cf.png} \caption{Patched FLANNEL Confusion Matrix}