Tree Classification with Multispectral Image Data and Convolutional Neural Networks
This repository contains the source code for the university project for the module "Data Science in Earth Observation".
This project presents the study on the classification of dominant tree species in Germany through deep learning techniques. Traditional methods rely on manual surveys, which are labor-intensive and time-consuming. However, advancements in remote sensing technologies offer the potential to automate the classification process. The Sentinel-2 L2A Multispectral data was extracted using the Google Earth Engine tool. The final extracted data is a patch of 5 x 5 pixels containing 10 bands (B2, B3, B4, B5, B6, B7, B8, B8A, B11, B12) x 3 seasons (summer, spring, and autumn). After cleaning up and removing missing data, 15657 samples were obtained. The data is then split into training, validation and test dataset and normalized.
The preprocessed data was then fed into the model for training. For this project, there were 3 models proposed to be used for the classification task. FullyConnectedNetwork is a multilayer perceptron with a classification head attached to it. ResNetModified is a CNN based architecture which takes the advantages of a ResNet50 model with pre-trained weights. Since the pre-trained model was trained with a 3 channel input whereas our dataset has a 30 channel input, the first and last layer of this pre-trained model was modified in order to fit the classification task described in this project. ResNetExtended is another CNN based approach which also utilizes the pre-trained ResNet50 model. In this model, a preprocessing layer with convolutional layers and an upsampling layer was added on top of the base ResNet model.
The performance of the models were unexpectedly not satisfactory. The FullyConnectedNetwork performed best with an accuracy of 21 % and weighted F1-score of 19 % on the test dataset. The CNN based models which were expected to perform better in image classification tasks only got an accuracy of 14 % and weighted F1-score of 15 %. Since the performance of the models were not optimal, there are some suggestions for improvements. The resampling of band data might help in the performance of the model. Currently bands 2, 3, 4, and 8 have a spatial resolution of 10 meters and other bands used in this project have a spatial resolution of 20 meters. Resampling all bands to the same resolution may help in the increase of the model performance. Another aspect that may help in the performance of the model is to also use aerial images and perform a late fusion of both models trained with aerial and Sentinel data. This has been proven to achieve better results in the article presented by Ahlswede et. al. (7).